INFO: Downloading File to /root/PM-DARTS2/...

Succeed: Total num: 57, size: 521,803,915. OK num: 57(download 57 objects).

average speed 316436000(byte/s)

1.652916(s) elapsed
INFO: Downloading succeed.
Network is under initialization...
Network successfully initialized.
WARN: ./requirements.txt not found, skip installing requirements.
Training with a single process on 1 GPUs.
Data processing configuration for current model + dataset:
	input_size: (3, 32, 32)
	interpolation: bilinear
	mean: (0.49139968, 0.48215827, 0.44653124)
	std: (0.24703233, 0.24348505, 0.26158768)
	crop_pct: 1.0
	crop_mode: center

-------------------------------
Learnable parameters
Student: 0.67M
Extra: 0.00M
-------------------------------
Scheduled epochs: 50
p_max: 0.2
search_space = s3
Using downloaded and verified file: /mnt/PM-DARTS2/data/cifar-10-python.tar.gz
Extracting /mnt/PM-DARTS2/data/cifar-10-python.tar.gz to /mnt/PM-DARTS2/data
Train: 0 [   0/390]  Loss: 2.278 (2.28)  Acc@1: 20.3125 (20.3125)  Acc@5: 62.5000 (62.5000)LR: 2.500e-02
Train: 0 [  50/390]  Loss: 1.863 (2.01)  Acc@1: 37.5000 (24.0809)  Acc@5: 81.2500 (76.8076)LR: 2.500e-02
Train: 0 [ 100/390]  Loss: 1.717 (1.90)  Acc@1: 39.0625 (28.4499)  Acc@5: 85.9375 (81.0953)LR: 2.500e-02
Train: 0 [ 150/390]  Loss: 1.668 (1.82)  Acc@1: 40.6250 (31.4156)  Acc@5: 87.5000 (83.3609)LR: 2.500e-02
Train: 0 [ 200/390]  Loss: 1.351 (1.76)  Acc@1: 46.8750 (33.6676)  Acc@5: 93.7500 (85.1835)LR: 2.500e-02
Train: 0 [ 250/390]  Loss: 1.657 (1.72)  Acc@1: 37.5000 (35.1407)  Acc@5: 90.6250 (86.3608)LR: 2.500e-02
Train: 0 [ 300/390]  Loss: 1.430 (1.67)  Acc@1: 54.6875 (37.1730)  Acc@5: 93.7500 (87.4792)LR: 2.500e-02
Train: 0 [ 350/390]  Loss: 1.327 (1.64)  Acc@1: 51.5625 (38.6797)  Acc@5: 96.8750 (88.1989)LR: 2.500e-02
Train: 0 [ 390/390]  Loss: 1.116 (1.61)  Acc@1: 65.0000 (39.8880)  Acc@5: 92.5000 (88.7880)LR: 2.500e-02
train_acc 39.888000
Valid: 0 [   0/390]  Loss: 1.364 (1.36)  Acc@1: 43.7500 (43.7500)  Acc@5: 95.3125 (95.3125)
Valid: 0 [  50/390]  Loss: 1.422 (1.36)  Acc@1: 48.4375 (51.0417)  Acc@5: 96.8750 (93.5662)
Valid: 0 [ 100/390]  Loss: 1.568 (1.36)  Acc@1: 42.1875 (50.5569)  Acc@5: 89.0625 (93.4870)
Valid: 0 [ 150/390]  Loss: 1.304 (1.37)  Acc@1: 54.6875 (50.4863)  Acc@5: 92.1875 (93.3671)
Valid: 0 [ 200/390]  Loss: 1.239 (1.36)  Acc@1: 48.4375 (50.7774)  Acc@5: 95.3125 (93.5012)
Valid: 0 [ 250/390]  Loss: 1.509 (1.37)  Acc@1: 37.5000 (50.4295)  Acc@5: 90.6250 (93.2520)
Valid: 0 [ 300/390]  Loss: 1.480 (1.37)  Acc@1: 40.6250 (50.3686)  Acc@5: 98.4375 (93.3295)
Valid: 0 [ 350/390]  Loss: 1.444 (1.37)  Acc@1: 45.3125 (50.3873)  Acc@5: 89.0625 (93.2470)
Valid: 0 [ 390/390]  Loss: 1.575 (1.37)  Acc@1: 37.5000 (50.2800)  Acc@5: 95.0000 (93.2040)
valid_acc 50.280000
epoch = 0   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('skip_connect', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('skip_connect', 4), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3364, 0.3300, 0.3336],
        [0.3375, 0.3296, 0.3329],
        [0.3356, 0.3262, 0.3381],
        [0.3281, 0.3309, 0.3410],
        [0.3361, 0.3277, 0.3362],
        [0.3382, 0.3286, 0.3332],
        [0.3379, 0.3321, 0.3300],
        [0.3361, 0.3270, 0.3368],
        [0.3323, 0.3286, 0.3391],
        [0.3371, 0.3294, 0.3335],
        [0.3324, 0.3292, 0.3384],
        [0.3380, 0.3293, 0.3327],
        [0.3304, 0.3278, 0.3418],
        [0.3369, 0.3274, 0.3357]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3343, 0.3324, 0.3332],
        [0.3321, 0.3355, 0.3323],
        [0.3341, 0.3346, 0.3312],
        [0.3357, 0.3342, 0.3301],
        [0.3328, 0.3321, 0.3350],
        [0.3344, 0.3294, 0.3363],
        [0.3346, 0.3319, 0.3336],
        [0.3325, 0.3327, 0.3348],
        [0.3299, 0.3297, 0.3404],
        [0.3312, 0.3338, 0.3350],
        [0.3348, 0.3347, 0.3305],
        [0.3353, 0.3335, 0.3312],
        [0.3339, 0.3321, 0.3340],
        [0.3320, 0.3370, 0.3309]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 1 [   0/390]  Loss: 1.366 (1.37)  Acc@1: 60.9375 (60.9375)  Acc@5: 92.1875 (92.1875)LR: 2.498e-02
Train: 1 [  50/390]  Loss: 1.199 (1.29)  Acc@1: 59.3750 (53.3088)  Acc@5: 95.3125 (94.4240)LR: 2.498e-02
Train: 1 [ 100/390]  Loss: 1.499 (1.28)  Acc@1: 45.3125 (53.2952)  Acc@5: 90.6250 (94.3688)LR: 2.498e-02
Train: 1 [ 150/390]  Loss: 1.569 (1.27)  Acc@1: 37.5000 (53.9321)  Acc@5: 90.6250 (94.3709)LR: 2.498e-02
Train: 1 [ 200/390]  Loss: 1.025 (1.26)  Acc@1: 60.9375 (54.3377)  Acc@5: 96.8750 (94.4496)LR: 2.498e-02
Train: 1 [ 250/390]  Loss: 1.248 (1.24)  Acc@1: 56.2500 (55.0299)  Acc@5: 93.7500 (94.6713)LR: 2.498e-02
Train: 1 [ 300/390]  Loss: 1.105 (1.23)  Acc@1: 62.5000 (55.6582)  Acc@5: 96.8750 (94.7778)LR: 2.498e-02
Train: 1 [ 350/390]  Loss: 1.150 (1.20)  Acc@1: 57.8125 (56.4904)  Acc@5: 98.4375 (95.1033)LR: 2.498e-02
Train: 1 [ 390/390]  Loss: 0.9543 (1.19)  Acc@1: 65.0000 (56.8880)  Acc@5: 95.0000 (95.2080)LR: 2.498e-02
train_acc 56.888000
Valid: 1 [   0/390]  Loss: 1.456 (1.46)  Acc@1: 46.8750 (46.8750)  Acc@5: 100.0000 (100.0000)
Valid: 1 [  50/390]  Loss: 1.238 (1.19)  Acc@1: 53.1250 (57.6900)  Acc@5: 96.8750 (94.9142)
Valid: 1 [ 100/390]  Loss: 1.264 (1.18)  Acc@1: 50.0000 (58.1064)  Acc@5: 93.7500 (95.2042)
Valid: 1 [ 150/390]  Loss: 1.211 (1.21)  Acc@1: 46.8750 (56.9847)  Acc@5: 95.3125 (95.0435)
Valid: 1 [ 200/390]  Loss: 1.226 (1.20)  Acc@1: 57.8125 (56.8252)  Acc@5: 98.4375 (95.1337)
Valid: 1 [ 250/390]  Loss: 1.265 (1.21)  Acc@1: 57.8125 (56.7729)  Acc@5: 95.3125 (95.0261)
Valid: 1 [ 300/390]  Loss: 1.237 (1.21)  Acc@1: 53.1250 (56.7639)  Acc@5: 95.3125 (95.0893)
Valid: 1 [ 350/390]  Loss: 1.246 (1.21)  Acc@1: 56.2500 (56.8064)  Acc@5: 92.1875 (95.1433)
Valid: 1 [ 390/390]  Loss: 1.241 (1.21)  Acc@1: 52.5000 (56.8520)  Acc@5: 95.0000 (95.1760)
valid_acc 56.852000
epoch = 1   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('skip_connect', 0), ('skip_connect', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3408, 0.3273, 0.3319],
        [0.3387, 0.3289, 0.3323],
        [0.3376, 0.3195, 0.3429],
        [0.3266, 0.3232, 0.3503],
        [0.3359, 0.3183, 0.3458],
        [0.3396, 0.3240, 0.3364],
        [0.3407, 0.3287, 0.3306],
        [0.3428, 0.3228, 0.3344],
        [0.3301, 0.3250, 0.3449],
        [0.3425, 0.3225, 0.3350],
        [0.3343, 0.3239, 0.3418],
        [0.3396, 0.3221, 0.3383],
        [0.3278, 0.3228, 0.3494],
        [0.3379, 0.3185, 0.3436]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3380, 0.3313, 0.3307],
        [0.3309, 0.3327, 0.3364],
        [0.3352, 0.3361, 0.3287],
        [0.3403, 0.3305, 0.3292],
        [0.3354, 0.3291, 0.3355],
        [0.3356, 0.3258, 0.3386],
        [0.3363, 0.3270, 0.3367],
        [0.3308, 0.3303, 0.3389],
        [0.3289, 0.3236, 0.3476],
        [0.3275, 0.3299, 0.3426],
        [0.3345, 0.3353, 0.3302],
        [0.3379, 0.3311, 0.3310],
        [0.3321, 0.3274, 0.3405],
        [0.3321, 0.3334, 0.3345]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 2 [   0/390]  Loss: 1.067 (1.07)  Acc@1: 62.5000 (62.5000)  Acc@5: 95.3125 (95.3125)LR: 2.491e-02
Train: 2 [  50/390]  Loss: 0.9405 (1.05)  Acc@1: 67.1875 (62.4694)  Acc@5: 98.4375 (96.2010)LR: 2.491e-02
Train: 2 [ 100/390]  Loss: 0.7899 (1.05)  Acc@1: 78.1250 (62.5309)  Acc@5: 96.8750 (96.2717)LR: 2.491e-02
Train: 2 [ 150/390]  Loss: 0.7350 (1.04)  Acc@1: 73.4375 (62.8311)  Acc@5: 98.4375 (96.4094)LR: 2.491e-02
Train: 2 [ 200/390]  Loss: 1.013 (1.02)  Acc@1: 64.0625 (63.1608)  Acc@5: 98.4375 (96.5330)LR: 2.491e-02
Train: 2 [ 250/390]  Loss: 0.9378 (1.02)  Acc@1: 67.1875 (63.3404)  Acc@5: 96.8750 (96.5762)LR: 2.491e-02
Train: 2 [ 300/390]  Loss: 0.9874 (1.01)  Acc@1: 64.0625 (63.5797)  Acc@5: 95.3125 (96.6466)LR: 2.491e-02
Train: 2 [ 350/390]  Loss: 0.9226 (1.00)  Acc@1: 64.0625 (63.9779)  Acc@5: 96.8750 (96.7325)LR: 2.491e-02
Train: 2 [ 390/390]  Loss: 0.9716 (0.996)  Acc@1: 67.5000 (64.2040)  Acc@5: 100.0000 (96.7600)LR: 2.491e-02
train_acc 64.204000
Valid: 2 [   0/390]  Loss: 0.9511 (0.951)  Acc@1: 65.6250 (65.6250)  Acc@5: 98.4375 (98.4375)
Valid: 2 [  50/390]  Loss: 0.9241 (0.987)  Acc@1: 68.7500 (64.1544)  Acc@5: 95.3125 (96.9975)
Valid: 2 [ 100/390]  Loss: 0.7986 (0.993)  Acc@1: 68.7500 (64.3410)  Acc@5: 96.8750 (96.8131)
Valid: 2 [ 150/390]  Loss: 0.8514 (0.982)  Acc@1: 67.1875 (64.4350)  Acc@5: 96.8750 (96.8750)
Valid: 2 [ 200/390]  Loss: 1.258 (0.978)  Acc@1: 53.1250 (64.8632)  Acc@5: 96.8750 (96.8595)
Valid: 2 [ 250/390]  Loss: 0.8415 (0.979)  Acc@1: 71.8750 (64.9963)  Acc@5: 98.4375 (96.8750)
Valid: 2 [ 300/390]  Loss: 1.067 (0.975)  Acc@1: 62.5000 (65.1215)  Acc@5: 96.8750 (96.8750)
Valid: 2 [ 350/390]  Loss: 0.8522 (0.978)  Acc@1: 59.3750 (65.0953)  Acc@5: 96.8750 (96.8305)
Valid: 2 [ 390/390]  Loss: 1.018 (0.982)  Acc@1: 65.0000 (64.9560)  Acc@5: 95.0000 (96.8200)
valid_acc 64.956000
epoch = 2   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('skip_connect', 0), ('skip_connect', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3416, 0.3291, 0.3293],
        [0.3430, 0.3276, 0.3295],
        [0.3384, 0.3137, 0.3479],
        [0.3252, 0.3193, 0.3556],
        [0.3396, 0.3148, 0.3456],
        [0.3374, 0.3235, 0.3392],
        [0.3435, 0.3225, 0.3340],
        [0.3484, 0.3204, 0.3312],
        [0.3309, 0.3215, 0.3476],
        [0.3435, 0.3198, 0.3367],
        [0.3369, 0.3193, 0.3438],
        [0.3418, 0.3187, 0.3395],
        [0.3241, 0.3162, 0.3597],
        [0.3384, 0.3115, 0.3501]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3386, 0.3318, 0.3296],
        [0.3304, 0.3322, 0.3374],
        [0.3339, 0.3386, 0.3275],
        [0.3407, 0.3304, 0.3289],
        [0.3396, 0.3275, 0.3329],
        [0.3364, 0.3246, 0.3390],
        [0.3335, 0.3265, 0.3400],
        [0.3327, 0.3294, 0.3379],
        [0.3278, 0.3238, 0.3484],
        [0.3267, 0.3265, 0.3469],
        [0.3348, 0.3306, 0.3346],
        [0.3415, 0.3314, 0.3272],
        [0.3323, 0.3255, 0.3422],
        [0.3327, 0.3294, 0.3380]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 3 [   0/390]  Loss: 0.8618 (0.862)  Acc@1: 67.1875 (67.1875)  Acc@5: 100.0000 (100.0000)LR: 2.479e-02
Train: 3 [  50/390]  Loss: 0.8928 (0.884)  Acc@1: 70.3125 (68.7194)  Acc@5: 98.4375 (97.7022)LR: 2.479e-02
Train: 3 [ 100/390]  Loss: 0.7127 (0.893)  Acc@1: 70.3125 (68.1621)  Acc@5: 98.4375 (97.5402)LR: 2.479e-02
Train: 3 [ 150/390]  Loss: 0.8257 (0.892)  Acc@1: 62.5000 (68.0153)  Acc@5: 100.0000 (97.6925)LR: 2.479e-02
Train: 3 [ 200/390]  Loss: 0.7016 (0.891)  Acc@1: 76.5625 (68.1903)  Acc@5: 96.8750 (97.6524)LR: 2.479e-02
Train: 3 [ 250/390]  Loss: 0.9524 (0.889)  Acc@1: 68.7500 (68.2022)  Acc@5: 95.3125 (97.5909)LR: 2.479e-02
Train: 3 [ 300/390]  Loss: 0.7805 (0.884)  Acc@1: 75.0000 (68.4333)  Acc@5: 95.3125 (97.6225)LR: 2.479e-02
Train: 3 [ 350/390]  Loss: 1.096 (0.880)  Acc@1: 68.7500 (68.7411)  Acc@5: 96.8750 (97.6763)LR: 2.479e-02
Train: 3 [ 390/390]  Loss: 0.9666 (0.875)  Acc@1: 62.5000 (68.9240)  Acc@5: 95.0000 (97.6880)LR: 2.479e-02
train_acc 68.924000
Valid: 3 [   0/390]  Loss: 1.283 (1.28)  Acc@1: 57.8125 (57.8125)  Acc@5: 92.1875 (92.1875)
Valid: 3 [  50/390]  Loss: 0.9229 (0.874)  Acc@1: 68.7500 (69.0564)  Acc@5: 96.8750 (97.3652)
Valid: 3 [ 100/390]  Loss: 0.8892 (0.878)  Acc@1: 62.5000 (69.3069)  Acc@5: 98.4375 (97.3236)
Valid: 3 [ 150/390]  Loss: 0.8444 (0.870)  Acc@1: 65.6250 (69.2984)  Acc@5: 95.3125 (97.3406)
Valid: 3 [ 200/390]  Loss: 0.9645 (0.870)  Acc@1: 67.1875 (69.0687)  Acc@5: 95.3125 (97.3570)
Valid: 3 [ 250/390]  Loss: 0.8270 (0.869)  Acc@1: 71.8750 (69.3103)  Acc@5: 98.4375 (97.3232)
Valid: 3 [ 300/390]  Loss: 0.8084 (0.872)  Acc@1: 75.0000 (69.2535)  Acc@5: 96.8750 (97.3630)
Valid: 3 [ 350/390]  Loss: 0.9980 (0.875)  Acc@1: 71.8750 (69.2174)  Acc@5: 98.4375 (97.3246)
Valid: 3 [ 390/390]  Loss: 0.8093 (0.876)  Acc@1: 62.5000 (69.1480)  Acc@5: 100.0000 (97.3240)
valid_acc 69.148000
epoch = 3   
 genotype = Genotype(normal=[('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3409, 0.3302, 0.3289],
        [0.3458, 0.3266, 0.3276],
        [0.3360, 0.3134, 0.3506],
        [0.3256, 0.3145, 0.3599],
        [0.3401, 0.3110, 0.3489],
        [0.3371, 0.3222, 0.3407],
        [0.3468, 0.3174, 0.3358],
        [0.3513, 0.3175, 0.3312],
        [0.3297, 0.3172, 0.3531],
        [0.3425, 0.3207, 0.3368],
        [0.3403, 0.3148, 0.3449],
        [0.3407, 0.3169, 0.3424],
        [0.3205, 0.3124, 0.3671],
        [0.3373, 0.3076, 0.3551]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3379, 0.3298, 0.3323],
        [0.3294, 0.3271, 0.3435],
        [0.3374, 0.3365, 0.3261],
        [0.3422, 0.3321, 0.3257],
        [0.3404, 0.3252, 0.3345],
        [0.3389, 0.3217, 0.3394],
        [0.3339, 0.3239, 0.3422],
        [0.3333, 0.3287, 0.3380],
        [0.3327, 0.3187, 0.3485],
        [0.3254, 0.3269, 0.3477],
        [0.3321, 0.3297, 0.3382],
        [0.3416, 0.3304, 0.3280],
        [0.3343, 0.3198, 0.3459],
        [0.3344, 0.3236, 0.3420]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 4 [   0/390]  Loss: 0.6232 (0.623)  Acc@1: 75.0000 (75.0000)  Acc@5: 98.4375 (98.4375)LR: 2.462e-02
Train: 4 [  50/390]  Loss: 0.7487 (0.798)  Acc@1: 71.8750 (71.4461)  Acc@5: 100.0000 (98.0699)LR: 2.462e-02
Train: 4 [ 100/390]  Loss: 0.6692 (0.802)  Acc@1: 79.6875 (71.5347)  Acc@5: 98.4375 (98.0353)LR: 2.462e-02
Train: 4 [ 150/390]  Loss: 1.071 (0.790)  Acc@1: 67.1875 (71.9474)  Acc@5: 95.3125 (98.0546)LR: 2.462e-02
Train: 4 [ 200/390]  Loss: 0.6692 (0.779)  Acc@1: 70.3125 (72.5824)  Acc@5: 100.0000 (98.1032)LR: 2.462e-02
Train: 4 [ 250/390]  Loss: 0.6077 (0.781)  Acc@1: 84.3750 (72.8772)  Acc@5: 98.4375 (97.9955)LR: 2.462e-02
Train: 4 [ 300/390]  Loss: 0.6281 (0.778)  Acc@1: 75.0000 (72.8821)  Acc@5: 95.3125 (98.0949)LR: 2.462e-02
Train: 4 [ 350/390]  Loss: 0.9393 (0.777)  Acc@1: 68.7500 (72.8632)  Acc@5: 95.3125 (98.0725)LR: 2.462e-02
Train: 4 [ 390/390]  Loss: 1.241 (0.774)  Acc@1: 67.5000 (73.0280)  Acc@5: 97.5000 (98.0880)LR: 2.462e-02
train_acc 73.028000
Valid: 4 [   0/390]  Loss: 0.4974 (0.497)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)
Valid: 4 [  50/390]  Loss: 0.6603 (0.747)  Acc@1: 82.8125 (74.6936)  Acc@5: 98.4375 (98.3150)
Valid: 4 [ 100/390]  Loss: 0.8020 (0.765)  Acc@1: 71.8750 (73.7469)  Acc@5: 100.0000 (98.0507)
Valid: 4 [ 150/390]  Loss: 0.8119 (0.765)  Acc@1: 70.3125 (73.5203)  Acc@5: 98.4375 (98.1788)
Valid: 4 [ 200/390]  Loss: 1.070 (0.762)  Acc@1: 65.6250 (73.4841)  Acc@5: 96.8750 (98.1965)
Valid: 4 [ 250/390]  Loss: 0.8161 (0.767)  Acc@1: 71.8750 (73.3254)  Acc@5: 96.8750 (98.1387)
Valid: 4 [ 300/390]  Loss: 0.9030 (0.769)  Acc@1: 64.0625 (73.1468)  Acc@5: 95.3125 (98.1001)
Valid: 4 [ 350/390]  Loss: 0.6733 (0.769)  Acc@1: 76.5625 (73.1214)  Acc@5: 100.0000 (98.1481)
Valid: 4 [ 390/390]  Loss: 1.034 (0.766)  Acc@1: 65.0000 (73.1840)  Acc@5: 97.5000 (98.1880)
valid_acc 73.184000
epoch = 4   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3419, 0.3288, 0.3293],
        [0.3487, 0.3234, 0.3279],
        [0.3352, 0.3122, 0.3526],
        [0.3244, 0.3126, 0.3631],
        [0.3420, 0.3107, 0.3472],
        [0.3381, 0.3196, 0.3423],
        [0.3498, 0.3119, 0.3383],
        [0.3511, 0.3143, 0.3346],
        [0.3321, 0.3152, 0.3526],
        [0.3446, 0.3208, 0.3346],
        [0.3478, 0.3133, 0.3388],
        [0.3404, 0.3130, 0.3466],
        [0.3181, 0.3075, 0.3744],
        [0.3445, 0.3064, 0.3491]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3406, 0.3264, 0.3331],
        [0.3221, 0.3250, 0.3529],
        [0.3397, 0.3326, 0.3277],
        [0.3422, 0.3342, 0.3236],
        [0.3399, 0.3263, 0.3339],
        [0.3436, 0.3191, 0.3373],
        [0.3319, 0.3262, 0.3419],
        [0.3325, 0.3299, 0.3376],
        [0.3362, 0.3195, 0.3443],
        [0.3249, 0.3243, 0.3508],
        [0.3314, 0.3278, 0.3408],
        [0.3391, 0.3344, 0.3265],
        [0.3331, 0.3189, 0.3480],
        [0.3366, 0.3218, 0.3417]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 5 [   0/390]  Loss: 0.7263 (0.726)  Acc@1: 73.4375 (73.4375)  Acc@5: 100.0000 (100.0000)LR: 2.441e-02
Train: 5 [  50/390]  Loss: 0.6084 (0.711)  Acc@1: 79.6875 (75.7659)  Acc@5: 100.0000 (98.5907)LR: 2.441e-02
Train: 5 [ 100/390]  Loss: 0.6324 (0.715)  Acc@1: 79.6875 (75.1547)  Acc@5: 96.8750 (98.5303)LR: 2.441e-02
Train: 5 [ 150/390]  Loss: 0.6770 (0.705)  Acc@1: 76.5625 (75.2587)  Acc@5: 98.4375 (98.6341)LR: 2.441e-02
Train: 5 [ 200/390]  Loss: 0.7267 (0.712)  Acc@1: 76.5625 (75.0000)  Acc@5: 96.8750 (98.5075)LR: 2.441e-02
Train: 5 [ 250/390]  Loss: 0.5716 (0.714)  Acc@1: 78.1250 (74.9564)  Acc@5: 100.0000 (98.4375)LR: 2.441e-02
Train: 5 [ 300/390]  Loss: 0.7488 (0.709)  Acc@1: 73.4375 (74.9792)  Acc@5: 96.8750 (98.4790)LR: 2.441e-02
Train: 5 [ 350/390]  Loss: 0.4703 (0.708)  Acc@1: 82.8125 (74.9644)  Acc@5: 98.4375 (98.4420)LR: 2.441e-02
Train: 5 [ 390/390]  Loss: 0.8194 (0.707)  Acc@1: 70.0000 (75.1040)  Acc@5: 100.0000 (98.4400)LR: 2.441e-02
train_acc 75.104000
Valid: 5 [   0/390]  Loss: 0.6590 (0.659)  Acc@1: 76.5625 (76.5625)  Acc@5: 98.4375 (98.4375)
Valid: 5 [  50/390]  Loss: 1.018 (0.750)  Acc@1: 76.5625 (74.0809)  Acc@5: 92.1875 (98.2230)
Valid: 5 [ 100/390]  Loss: 0.8743 (0.752)  Acc@1: 75.0000 (73.7933)  Acc@5: 96.8750 (98.2828)
Valid: 5 [ 150/390]  Loss: 0.9223 (0.749)  Acc@1: 67.1875 (73.9342)  Acc@5: 96.8750 (98.2409)
Valid: 5 [ 200/390]  Loss: 0.5398 (0.744)  Acc@1: 81.2500 (74.0905)  Acc@5: 100.0000 (98.3131)
Valid: 5 [ 250/390]  Loss: 0.7177 (0.738)  Acc@1: 75.0000 (74.3526)  Acc@5: 100.0000 (98.3877)
Valid: 5 [ 300/390]  Loss: 0.5917 (0.733)  Acc@1: 76.5625 (74.4549)  Acc@5: 100.0000 (98.4427)
Valid: 5 [ 350/390]  Loss: 0.5447 (0.729)  Acc@1: 78.1250 (74.6261)  Acc@5: 100.0000 (98.4420)
Valid: 5 [ 390/390]  Loss: 0.5597 (0.728)  Acc@1: 82.5000 (74.6520)  Acc@5: 97.5000 (98.4480)
valid_acc 74.652000
epoch = 5   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3430, 0.3244, 0.3326],
        [0.3508, 0.3186, 0.3306],
        [0.3401, 0.3075, 0.3524],
        [0.3202, 0.3089, 0.3709],
        [0.3418, 0.3099, 0.3483],
        [0.3441, 0.3135, 0.3424],
        [0.3524, 0.3057, 0.3418],
        [0.3517, 0.3096, 0.3387],
        [0.3309, 0.3107, 0.3584],
        [0.3495, 0.3158, 0.3346],
        [0.3520, 0.3082, 0.3398],
        [0.3430, 0.3132, 0.3438],
        [0.3162, 0.3036, 0.3802],
        [0.3493, 0.3003, 0.3504]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3414, 0.3245, 0.3341],
        [0.3210, 0.3256, 0.3534],
        [0.3436, 0.3259, 0.3304],
        [0.3439, 0.3330, 0.3231],
        [0.3439, 0.3262, 0.3299],
        [0.3448, 0.3168, 0.3384],
        [0.3333, 0.3269, 0.3398],
        [0.3304, 0.3292, 0.3405],
        [0.3388, 0.3155, 0.3456],
        [0.3231, 0.3229, 0.3540],
        [0.3300, 0.3246, 0.3454],
        [0.3362, 0.3315, 0.3323],
        [0.3338, 0.3174, 0.3488],
        [0.3318, 0.3203, 0.3479]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 6 [   0/390]  Loss: 0.5638 (0.564)  Acc@1: 75.0000 (75.0000)  Acc@5: 100.0000 (100.0000)LR: 2.416e-02
Train: 6 [  50/390]  Loss: 0.8120 (0.666)  Acc@1: 71.8750 (76.2255)  Acc@5: 95.3125 (98.7439)LR: 2.416e-02
Train: 6 [ 100/390]  Loss: 0.5801 (0.664)  Acc@1: 82.8125 (76.6553)  Acc@5: 98.4375 (98.6850)LR: 2.416e-02
Train: 6 [ 150/390]  Loss: 0.6770 (0.659)  Acc@1: 73.4375 (76.8419)  Acc@5: 98.4375 (98.6858)LR: 2.416e-02
Train: 6 [ 200/390]  Loss: 0.8208 (0.657)  Acc@1: 70.3125 (77.0989)  Acc@5: 98.4375 (98.6707)LR: 2.416e-02
Train: 6 [ 250/390]  Loss: 1.092 (0.658)  Acc@1: 62.5000 (77.0605)  Acc@5: 96.8750 (98.6803)LR: 2.416e-02
Train: 6 [ 300/390]  Loss: 0.7949 (0.661)  Acc@1: 68.7500 (77.0660)  Acc@5: 98.4375 (98.6503)LR: 2.416e-02
Train: 6 [ 350/390]  Loss: 0.5350 (0.662)  Acc@1: 79.6875 (77.0878)  Acc@5: 100.0000 (98.6779)LR: 2.416e-02
Train: 6 [ 390/390]  Loss: 0.7435 (0.664)  Acc@1: 80.0000 (76.9800)  Acc@5: 95.0000 (98.6320)LR: 2.416e-02
train_acc 76.980000
Valid: 6 [   0/390]  Loss: 0.6104 (0.610)  Acc@1: 84.3750 (84.3750)  Acc@5: 96.8750 (96.8750)
Valid: 6 [  50/390]  Loss: 0.7749 (0.747)  Acc@1: 75.0000 (74.0809)  Acc@5: 98.4375 (97.9167)
Valid: 6 [ 100/390]  Loss: 0.7952 (0.755)  Acc@1: 75.0000 (73.9790)  Acc@5: 100.0000 (97.9425)
Valid: 6 [ 150/390]  Loss: 0.8654 (0.759)  Acc@1: 68.7500 (73.8100)  Acc@5: 98.4375 (97.9615)
Valid: 6 [ 200/390]  Loss: 1.229 (0.768)  Acc@1: 65.6250 (73.5075)  Acc@5: 96.8750 (97.8467)
Valid: 6 [ 250/390]  Loss: 0.9978 (0.761)  Acc@1: 73.4375 (73.6990)  Acc@5: 95.3125 (97.9333)
Valid: 6 [ 300/390]  Loss: 0.6225 (0.761)  Acc@1: 75.0000 (73.6971)  Acc@5: 98.4375 (97.9392)
Valid: 6 [ 350/390]  Loss: 1.030 (0.760)  Acc@1: 65.6250 (73.6912)  Acc@5: 98.4375 (97.9701)
Valid: 6 [ 390/390]  Loss: 0.5579 (0.756)  Acc@1: 87.5000 (73.8160)  Acc@5: 100.0000 (98.0120)
valid_acc 73.816000
epoch = 6   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3437, 0.3224, 0.3339],
        [0.3539, 0.3171, 0.3290],
        [0.3381, 0.3067, 0.3552],
        [0.3237, 0.3076, 0.3687],
        [0.3457, 0.3106, 0.3437],
        [0.3476, 0.3117, 0.3408],
        [0.3550, 0.2995, 0.3456],
        [0.3555, 0.3056, 0.3389],
        [0.3354, 0.3064, 0.3583],
        [0.3520, 0.3116, 0.3364],
        [0.3592, 0.3039, 0.3369],
        [0.3501, 0.3119, 0.3380],
        [0.3194, 0.2985, 0.3820],
        [0.3553, 0.2937, 0.3511]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3442, 0.3207, 0.3350],
        [0.3172, 0.3246, 0.3582],
        [0.3471, 0.3229, 0.3301],
        [0.3414, 0.3342, 0.3243],
        [0.3457, 0.3265, 0.3278],
        [0.3469, 0.3140, 0.3391],
        [0.3304, 0.3276, 0.3419],
        [0.3296, 0.3288, 0.3416],
        [0.3415, 0.3115, 0.3470],
        [0.3271, 0.3189, 0.3540],
        [0.3268, 0.3217, 0.3515],
        [0.3343, 0.3317, 0.3341],
        [0.3310, 0.3214, 0.3475],
        [0.3298, 0.3241, 0.3461]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 7 [   0/390]  Loss: 0.4803 (0.480)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)LR: 2.386e-02
Train: 7 [  50/390]  Loss: 0.4439 (0.579)  Acc@1: 90.6250 (80.1164)  Acc@5: 98.4375 (99.0502)LR: 2.386e-02
Train: 7 [ 100/390]  Loss: 0.6277 (0.608)  Acc@1: 76.5625 (78.7902)  Acc@5: 100.0000 (98.9635)LR: 2.386e-02
Train: 7 [ 150/390]  Loss: 0.6292 (0.602)  Acc@1: 73.4375 (79.1287)  Acc@5: 95.3125 (98.8825)LR: 2.386e-02
Train: 7 [ 200/390]  Loss: 0.5707 (0.614)  Acc@1: 79.6875 (78.7702)  Acc@5: 100.0000 (98.8340)LR: 2.386e-02
Train: 7 [ 250/390]  Loss: 0.5571 (0.609)  Acc@1: 82.8125 (78.8845)  Acc@5: 98.4375 (98.8484)LR: 2.386e-02
Train: 7 [ 300/390]  Loss: 0.7900 (0.611)  Acc@1: 78.1250 (78.7687)  Acc@5: 95.3125 (98.8320)LR: 2.386e-02
Train: 7 [ 350/390]  Loss: 0.5359 (0.613)  Acc@1: 76.5625 (78.6948)  Acc@5: 98.4375 (98.7847)LR: 2.386e-02
Train: 7 [ 390/390]  Loss: 0.7619 (0.617)  Acc@1: 72.5000 (78.6120)  Acc@5: 100.0000 (98.7840)LR: 2.386e-02
train_acc 78.612000
Valid: 7 [   0/390]  Loss: 0.5843 (0.584)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)
Valid: 7 [  50/390]  Loss: 0.6759 (0.644)  Acc@1: 73.4375 (77.6042)  Acc@5: 100.0000 (98.8358)
Valid: 7 [ 100/390]  Loss: 0.6844 (0.649)  Acc@1: 73.4375 (77.4752)  Acc@5: 100.0000 (98.8088)
Valid: 7 [ 150/390]  Loss: 0.5836 (0.647)  Acc@1: 85.9375 (77.6387)  Acc@5: 100.0000 (98.7376)
Valid: 7 [ 200/390]  Loss: 0.6559 (0.653)  Acc@1: 79.6875 (77.3321)  Acc@5: 98.4375 (98.6396)
Valid: 7 [ 250/390]  Loss: 0.4745 (0.653)  Acc@1: 85.9375 (77.2722)  Acc@5: 98.4375 (98.6243)
Valid: 7 [ 300/390]  Loss: 0.8574 (0.650)  Acc@1: 70.3125 (77.4190)  Acc@5: 96.8750 (98.6555)
Valid: 7 [ 350/390]  Loss: 0.7076 (0.648)  Acc@1: 78.1250 (77.3860)  Acc@5: 100.0000 (98.6957)
Valid: 7 [ 390/390]  Loss: 0.7543 (0.651)  Acc@1: 72.5000 (77.3680)  Acc@5: 97.5000 (98.7000)
valid_acc 77.368000
epoch = 7   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3460, 0.3225, 0.3315],
        [0.3545, 0.3113, 0.3343],
        [0.3371, 0.3065, 0.3564],
        [0.3273, 0.3031, 0.3696],
        [0.3456, 0.3093, 0.3452],
        [0.3487, 0.3114, 0.3400],
        [0.3616, 0.2944, 0.3440],
        [0.3592, 0.3040, 0.3367],
        [0.3375, 0.3036, 0.3589],
        [0.3540, 0.3113, 0.3347],
        [0.3637, 0.2999, 0.3365],
        [0.3541, 0.3099, 0.3360],
        [0.3212, 0.2959, 0.3829],
        [0.3633, 0.2889, 0.3479]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3472, 0.3157, 0.3371],
        [0.3131, 0.3232, 0.3637],
        [0.3480, 0.3218, 0.3302],
        [0.3405, 0.3342, 0.3253],
        [0.3472, 0.3247, 0.3281],
        [0.3483, 0.3140, 0.3377],
        [0.3281, 0.3290, 0.3429],
        [0.3295, 0.3273, 0.3432],
        [0.3421, 0.3100, 0.3480],
        [0.3318, 0.3197, 0.3485],
        [0.3238, 0.3217, 0.3545],
        [0.3355, 0.3330, 0.3314],
        [0.3291, 0.3227, 0.3482],
        [0.3324, 0.3252, 0.3424]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 8 [   0/390]  Loss: 0.6342 (0.634)  Acc@1: 79.6875 (79.6875)  Acc@5: 96.8750 (96.8750)LR: 2.352e-02
Train: 8 [  50/390]  Loss: 0.4761 (0.618)  Acc@1: 85.9375 (79.2892)  Acc@5: 98.4375 (98.6826)LR: 2.352e-02
Train: 8 [ 100/390]  Loss: 0.6240 (0.608)  Acc@1: 82.8125 (78.9913)  Acc@5: 98.4375 (98.7314)LR: 2.352e-02
Train: 8 [ 150/390]  Loss: 0.4469 (0.609)  Acc@1: 81.2500 (78.9218)  Acc@5: 100.0000 (98.7893)LR: 2.352e-02
Train: 8 [ 200/390]  Loss: 0.6657 (0.602)  Acc@1: 81.2500 (79.0812)  Acc@5: 96.8750 (98.8495)LR: 2.352e-02
Train: 8 [ 250/390]  Loss: 0.6002 (0.604)  Acc@1: 76.5625 (78.9280)  Acc@5: 100.0000 (98.8608)LR: 2.352e-02
Train: 8 [ 300/390]  Loss: 0.8036 (0.603)  Acc@1: 70.3125 (78.8829)  Acc@5: 93.7500 (98.8839)LR: 2.352e-02
Train: 8 [ 350/390]  Loss: 0.4748 (0.599)  Acc@1: 82.8125 (79.0554)  Acc@5: 100.0000 (98.8782)LR: 2.352e-02
Train: 8 [ 390/390]  Loss: 0.5556 (0.597)  Acc@1: 75.0000 (79.0800)  Acc@5: 100.0000 (98.8760)LR: 2.352e-02
train_acc 79.080000
Valid: 8 [   0/390]  Loss: 0.5262 (0.526)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 8 [  50/390]  Loss: 0.6080 (0.630)  Acc@1: 79.6875 (77.3284)  Acc@5: 98.4375 (98.7439)
Valid: 8 [ 100/390]  Loss: 0.5933 (0.617)  Acc@1: 76.5625 (78.0012)  Acc@5: 100.0000 (99.1182)
Valid: 8 [ 150/390]  Loss: 0.7071 (0.618)  Acc@1: 75.0000 (78.3733)  Acc@5: 96.8750 (98.9342)
Valid: 8 [ 200/390]  Loss: 0.7004 (0.621)  Acc@1: 73.4375 (78.5059)  Acc@5: 98.4375 (98.8806)
Valid: 8 [ 250/390]  Loss: 0.6792 (0.620)  Acc@1: 76.5625 (78.5234)  Acc@5: 100.0000 (98.8546)
Valid: 8 [ 300/390]  Loss: 0.3947 (0.619)  Acc@1: 87.5000 (78.6285)  Acc@5: 100.0000 (98.8216)
Valid: 8 [ 350/390]  Loss: 0.5752 (0.621)  Acc@1: 79.6875 (78.5613)  Acc@5: 100.0000 (98.7803)
Valid: 8 [ 390/390]  Loss: 0.2478 (0.623)  Acc@1: 95.0000 (78.5080)  Acc@5: 100.0000 (98.7160)
valid_acc 78.508000
epoch = 8   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3485, 0.3205, 0.3310],
        [0.3573, 0.3064, 0.3363],
        [0.3383, 0.3033, 0.3584],
        [0.3268, 0.3012, 0.3720],
        [0.3487, 0.3075, 0.3438],
        [0.3489, 0.3054, 0.3457],
        [0.3674, 0.2877, 0.3449],
        [0.3635, 0.2995, 0.3370],
        [0.3432, 0.2985, 0.3583],
        [0.3581, 0.3075, 0.3345],
        [0.3681, 0.2966, 0.3353],
        [0.3592, 0.3078, 0.3330],
        [0.3232, 0.2915, 0.3853],
        [0.3705, 0.2837, 0.3458]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3471, 0.3134, 0.3395],
        [0.3129, 0.3215, 0.3656],
        [0.3477, 0.3207, 0.3316],
        [0.3370, 0.3365, 0.3265],
        [0.3514, 0.3216, 0.3270],
        [0.3489, 0.3152, 0.3359],
        [0.3276, 0.3288, 0.3436],
        [0.3306, 0.3278, 0.3415],
        [0.3445, 0.3099, 0.3456],
        [0.3342, 0.3176, 0.3482],
        [0.3213, 0.3188, 0.3598],
        [0.3350, 0.3290, 0.3360],
        [0.3273, 0.3274, 0.3453],
        [0.3359, 0.3224, 0.3417]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 9 [   0/390]  Loss: 0.4916 (0.492)  Acc@1: 85.9375 (85.9375)  Acc@5: 96.8750 (96.8750)LR: 2.313e-02
Train: 9 [  50/390]  Loss: 0.5590 (0.585)  Acc@1: 79.6875 (79.6569)  Acc@5: 98.4375 (98.7132)LR: 2.313e-02
Train: 9 [ 100/390]  Loss: 0.7525 (0.566)  Acc@1: 71.8750 (80.4920)  Acc@5: 100.0000 (98.9480)LR: 2.313e-02
Train: 9 [ 150/390]  Loss: 0.4336 (0.567)  Acc@1: 85.9375 (80.4429)  Acc@5: 100.0000 (99.0584)LR: 2.313e-02
Train: 9 [ 200/390]  Loss: 0.6510 (0.565)  Acc@1: 78.1250 (80.5115)  Acc@5: 98.4375 (99.0127)LR: 2.313e-02
Train: 9 [ 250/390]  Loss: 0.6858 (0.572)  Acc@1: 76.5625 (80.2229)  Acc@5: 100.0000 (99.0227)LR: 2.313e-02
Train: 9 [ 300/390]  Loss: 0.6152 (0.571)  Acc@1: 79.6875 (80.2533)  Acc@5: 98.4375 (99.0241)LR: 2.313e-02
Train: 9 [ 350/390]  Loss: 0.7362 (0.571)  Acc@1: 78.1250 (80.2974)  Acc@5: 98.4375 (99.0296)LR: 2.313e-02
Train: 9 [ 390/390]  Loss: 0.4063 (0.570)  Acc@1: 87.5000 (80.2640)  Acc@5: 100.0000 (99.0360)LR: 2.313e-02
train_acc 80.264000
Valid: 9 [   0/390]  Loss: 0.5547 (0.555)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)
Valid: 9 [  50/390]  Loss: 0.7268 (0.589)  Acc@1: 75.0000 (79.8407)  Acc@5: 95.3125 (98.8358)
Valid: 9 [ 100/390]  Loss: 0.5539 (0.599)  Acc@1: 81.2500 (79.3626)  Acc@5: 100.0000 (98.9171)
Valid: 9 [ 150/390]  Loss: 0.3826 (0.602)  Acc@1: 84.3750 (79.0977)  Acc@5: 100.0000 (98.8307)
Valid: 9 [ 200/390]  Loss: 0.6519 (0.609)  Acc@1: 76.5625 (78.8013)  Acc@5: 98.4375 (98.7484)
Valid: 9 [ 250/390]  Loss: 0.6558 (0.607)  Acc@1: 79.6875 (78.8782)  Acc@5: 100.0000 (98.7612)
Valid: 9 [ 300/390]  Loss: 0.5680 (0.605)  Acc@1: 76.5625 (78.9244)  Acc@5: 100.0000 (98.8061)
Valid: 9 [ 350/390]  Loss: 0.6628 (0.604)  Acc@1: 76.5625 (78.8328)  Acc@5: 98.4375 (98.7981)
Valid: 9 [ 390/390]  Loss: 0.5527 (0.601)  Acc@1: 85.0000 (79.0720)  Acc@5: 100.0000 (98.7920)
valid_acc 79.072000
epoch = 9   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3492, 0.3182, 0.3327],
        [0.3610, 0.3023, 0.3367],
        [0.3388, 0.3052, 0.3561],
        [0.3310, 0.2972, 0.3718],
        [0.3515, 0.3059, 0.3425],
        [0.3500, 0.3015, 0.3485],
        [0.3748, 0.2832, 0.3420],
        [0.3676, 0.2958, 0.3366],
        [0.3460, 0.2927, 0.3613],
        [0.3573, 0.3054, 0.3373],
        [0.3714, 0.2935, 0.3351],
        [0.3646, 0.3073, 0.3281],
        [0.3288, 0.2894, 0.3818],
        [0.3789, 0.2786, 0.3426]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3504, 0.3102, 0.3394],
        [0.3063, 0.3183, 0.3753],
        [0.3536, 0.3189, 0.3274],
        [0.3356, 0.3351, 0.3292],
        [0.3511, 0.3231, 0.3258],
        [0.3502, 0.3168, 0.3330],
        [0.3263, 0.3325, 0.3412],
        [0.3310, 0.3279, 0.3411],
        [0.3463, 0.3078, 0.3459],
        [0.3352, 0.3159, 0.3489],
        [0.3217, 0.3170, 0.3613],
        [0.3320, 0.3305, 0.3375],
        [0.3292, 0.3252, 0.3456],
        [0.3351, 0.3211, 0.3439]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 10 [   0/390]  Loss: 0.4773 (0.477)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)LR: 2.271e-02
Train: 10 [  50/390]  Loss: 0.4727 (0.536)  Acc@1: 81.2500 (81.6176)  Acc@5: 100.0000 (99.0196)LR: 2.271e-02
Train: 10 [ 100/390]  Loss: 0.5179 (0.533)  Acc@1: 79.6875 (81.5130)  Acc@5: 100.0000 (99.1337)LR: 2.271e-02
Train: 10 [ 150/390]  Loss: 0.5239 (0.528)  Acc@1: 82.8125 (81.6329)  Acc@5: 98.4375 (99.1308)LR: 2.271e-02
Train: 10 [ 200/390]  Loss: 0.5677 (0.537)  Acc@1: 78.1250 (81.4288)  Acc@5: 100.0000 (99.1838)LR: 2.271e-02
Train: 10 [ 250/390]  Loss: 0.6989 (0.540)  Acc@1: 75.0000 (81.2687)  Acc@5: 100.0000 (99.2156)LR: 2.271e-02
Train: 10 [ 300/390]  Loss: 0.5625 (0.542)  Acc@1: 76.5625 (81.2604)  Acc@5: 98.4375 (99.1954)LR: 2.271e-02
Train: 10 [ 350/390]  Loss: 0.5127 (0.539)  Acc@1: 78.1250 (81.3791)  Acc@5: 100.0000 (99.1898)LR: 2.271e-02
Train: 10 [ 390/390]  Loss: 0.2992 (0.537)  Acc@1: 95.0000 (81.5280)  Acc@5: 100.0000 (99.1960)LR: 2.271e-02
train_acc 81.528000
Valid: 10 [   0/390]  Loss: 0.4806 (0.481)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 10 [  50/390]  Loss: 0.4939 (0.565)  Acc@1: 76.5625 (80.2390)  Acc@5: 98.4375 (98.7439)
Valid: 10 [ 100/390]  Loss: 0.6055 (0.573)  Acc@1: 78.1250 (80.3527)  Acc@5: 98.4375 (98.7160)
Valid: 10 [ 150/390]  Loss: 0.6676 (0.572)  Acc@1: 79.6875 (80.4636)  Acc@5: 100.0000 (98.7997)
Valid: 10 [ 200/390]  Loss: 0.7096 (0.567)  Acc@1: 73.4375 (80.5737)  Acc@5: 96.8750 (98.8417)
Valid: 10 [ 250/390]  Loss: 0.5945 (0.567)  Acc@1: 82.8125 (80.5777)  Acc@5: 98.4375 (98.8110)
Valid: 10 [ 300/390]  Loss: 0.6600 (0.576)  Acc@1: 76.5625 (80.2949)  Acc@5: 100.0000 (98.7593)
Valid: 10 [ 350/390]  Loss: 0.4630 (0.574)  Acc@1: 87.5000 (80.3552)  Acc@5: 98.4375 (98.8114)
Valid: 10 [ 390/390]  Loss: 0.8824 (0.573)  Acc@1: 65.0000 (80.3760)  Acc@5: 100.0000 (98.8120)
valid_acc 80.376000
epoch = 10   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3495, 0.3190, 0.3315],
        [0.3671, 0.2977, 0.3353],
        [0.3386, 0.3076, 0.3538],
        [0.3387, 0.2928, 0.3685],
        [0.3558, 0.3041, 0.3401],
        [0.3507, 0.2997, 0.3497],
        [0.3831, 0.2808, 0.3361],
        [0.3730, 0.2939, 0.3331],
        [0.3548, 0.2882, 0.3570],
        [0.3586, 0.3047, 0.3367],
        [0.3787, 0.2892, 0.3321],
        [0.3714, 0.3057, 0.3229],
        [0.3329, 0.2851, 0.3820],
        [0.3905, 0.2708, 0.3386]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3512, 0.3075, 0.3413],
        [0.3039, 0.3161, 0.3799],
        [0.3568, 0.3175, 0.3256],
        [0.3369, 0.3314, 0.3317],
        [0.3539, 0.3223, 0.3238],
        [0.3496, 0.3176, 0.3328],
        [0.3199, 0.3374, 0.3426],
        [0.3306, 0.3268, 0.3426],
        [0.3516, 0.3023, 0.3461],
        [0.3366, 0.3170, 0.3464],
        [0.3212, 0.3192, 0.3597],
        [0.3322, 0.3278, 0.3401],
        [0.3296, 0.3259, 0.3445],
        [0.3364, 0.3228, 0.3408]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 11 [   0/390]  Loss: 0.5295 (0.529)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)LR: 2.225e-02
Train: 11 [  50/390]  Loss: 0.7465 (0.523)  Acc@1: 76.5625 (81.5564)  Acc@5: 98.4375 (99.1422)LR: 2.225e-02
Train: 11 [ 100/390]  Loss: 0.5217 (0.518)  Acc@1: 82.8125 (81.7915)  Acc@5: 98.4375 (99.1337)LR: 2.225e-02
Train: 11 [ 150/390]  Loss: 0.4481 (0.512)  Acc@1: 81.2500 (81.9226)  Acc@5: 100.0000 (99.1515)LR: 2.225e-02
Train: 11 [ 200/390]  Loss: 0.3383 (0.515)  Acc@1: 85.9375 (81.8408)  Acc@5: 100.0000 (99.2149)LR: 2.225e-02
Train: 11 [ 250/390]  Loss: 0.6106 (0.518)  Acc@1: 79.6875 (81.9348)  Acc@5: 98.4375 (99.1721)LR: 2.225e-02
Train: 11 [ 300/390]  Loss: 0.3622 (0.518)  Acc@1: 89.0625 (81.9093)  Acc@5: 100.0000 (99.1539)LR: 2.225e-02
Train: 11 [ 350/390]  Loss: 0.4163 (0.521)  Acc@1: 87.5000 (81.8732)  Acc@5: 100.0000 (99.1631)LR: 2.225e-02
Train: 11 [ 390/390]  Loss: 0.4583 (0.524)  Acc@1: 87.5000 (81.8240)  Acc@5: 100.0000 (99.1360)LR: 2.225e-02
train_acc 81.824000
Valid: 11 [   0/390]  Loss: 0.7114 (0.711)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)
Valid: 11 [  50/390]  Loss: 0.5274 (0.596)  Acc@1: 81.2500 (80.7598)  Acc@5: 100.0000 (98.8664)
Valid: 11 [ 100/390]  Loss: 0.5776 (0.583)  Acc@1: 82.8125 (80.3063)  Acc@5: 98.4375 (98.9171)
Valid: 11 [ 150/390]  Loss: 0.6383 (0.576)  Acc@1: 81.2500 (80.5257)  Acc@5: 93.7500 (98.9342)
Valid: 11 [ 200/390]  Loss: 0.6499 (0.581)  Acc@1: 75.0000 (80.3638)  Acc@5: 96.8750 (98.8806)
Valid: 11 [ 250/390]  Loss: 0.5528 (0.576)  Acc@1: 81.2500 (80.4345)  Acc@5: 100.0000 (98.8795)
Valid: 11 [ 300/390]  Loss: 0.6482 (0.576)  Acc@1: 84.3750 (80.4350)  Acc@5: 98.4375 (98.8684)
Valid: 11 [ 350/390]  Loss: 0.4855 (0.579)  Acc@1: 89.0625 (80.2172)  Acc@5: 98.4375 (98.8515)
Valid: 11 [ 390/390]  Loss: 0.3150 (0.579)  Acc@1: 87.5000 (80.1920)  Acc@5: 100.0000 (98.8640)
valid_acc 80.192000
epoch = 11   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3516, 0.3193, 0.3291],
        [0.3706, 0.2931, 0.3363],
        [0.3405, 0.3099, 0.3495],
        [0.3427, 0.2906, 0.3667],
        [0.3590, 0.3031, 0.3380],
        [0.3515, 0.2974, 0.3511],
        [0.3894, 0.2795, 0.3312],
        [0.3805, 0.2925, 0.3269],
        [0.3628, 0.2854, 0.3518],
        [0.3628, 0.3027, 0.3345],
        [0.3831, 0.2872, 0.3297],
        [0.3786, 0.3047, 0.3167],
        [0.3378, 0.2817, 0.3805],
        [0.3996, 0.2643, 0.3361]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3555, 0.3053, 0.3393],
        [0.3006, 0.3173, 0.3821],
        [0.3592, 0.3163, 0.3245],
        [0.3336, 0.3313, 0.3351],
        [0.3579, 0.3221, 0.3199],
        [0.3516, 0.3157, 0.3327],
        [0.3172, 0.3382, 0.3447],
        [0.3299, 0.3256, 0.3445],
        [0.3512, 0.2997, 0.3491],
        [0.3361, 0.3178, 0.3461],
        [0.3212, 0.3216, 0.3572],
        [0.3328, 0.3262, 0.3410],
        [0.3301, 0.3266, 0.3433],
        [0.3357, 0.3252, 0.3391]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 12 [   0/390]  Loss: 0.4178 (0.418)  Acc@1: 89.0625 (89.0625)  Acc@5: 98.4375 (98.4375)LR: 2.175e-02
Train: 12 [  50/390]  Loss: 0.5018 (0.478)  Acc@1: 79.6875 (83.8235)  Acc@5: 98.4375 (99.1422)LR: 2.175e-02
Train: 12 [ 100/390]  Loss: 0.2752 (0.495)  Acc@1: 93.7500 (83.0600)  Acc@5: 100.0000 (99.1801)LR: 2.175e-02
Train: 12 [ 150/390]  Loss: 0.4339 (0.506)  Acc@1: 81.2500 (82.5745)  Acc@5: 98.4375 (99.1515)LR: 2.175e-02
Train: 12 [ 200/390]  Loss: 0.5346 (0.500)  Acc@1: 81.2500 (82.6726)  Acc@5: 100.0000 (99.1760)LR: 2.175e-02
Train: 12 [ 250/390]  Loss: 0.4245 (0.501)  Acc@1: 87.5000 (82.6693)  Acc@5: 100.0000 (99.2032)LR: 2.175e-02
Train: 12 [ 300/390]  Loss: 0.7072 (0.500)  Acc@1: 79.6875 (82.7191)  Acc@5: 98.4375 (99.2369)LR: 2.175e-02
Train: 12 [ 350/390]  Loss: 0.6860 (0.502)  Acc@1: 81.2500 (82.5766)  Acc@5: 100.0000 (99.2566)LR: 2.175e-02
Train: 12 [ 390/390]  Loss: 0.6269 (0.502)  Acc@1: 77.5000 (82.6400)  Acc@5: 100.0000 (99.2600)LR: 2.175e-02
train_acc 82.640000
Valid: 12 [   0/390]  Loss: 0.7361 (0.736)  Acc@1: 68.7500 (68.7500)  Acc@5: 98.4375 (98.4375)
Valid: 12 [  50/390]  Loss: 0.3876 (0.558)  Acc@1: 89.0625 (81.0968)  Acc@5: 100.0000 (99.1115)
Valid: 12 [ 100/390]  Loss: 0.5168 (0.550)  Acc@1: 79.6875 (81.4511)  Acc@5: 98.4375 (99.1027)
Valid: 12 [ 150/390]  Loss: 0.6935 (0.551)  Acc@1: 78.1250 (81.2500)  Acc@5: 100.0000 (98.9238)
Valid: 12 [ 200/390]  Loss: 0.4069 (0.553)  Acc@1: 85.9375 (81.1956)  Acc@5: 100.0000 (98.9583)
Valid: 12 [ 250/390]  Loss: 0.8790 (0.551)  Acc@1: 67.1875 (81.2811)  Acc@5: 98.4375 (99.0040)
Valid: 12 [ 300/390]  Loss: 0.4206 (0.553)  Acc@1: 89.0625 (81.1306)  Acc@5: 98.4375 (98.9722)
Valid: 12 [ 350/390]  Loss: 0.6308 (0.554)  Acc@1: 81.2500 (81.0719)  Acc@5: 98.4375 (98.9850)
Valid: 12 [ 390/390]  Loss: 0.6889 (0.555)  Acc@1: 77.5000 (81.0200)  Acc@5: 97.5000 (98.9400)
valid_acc 81.020000
epoch = 12   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3517, 0.3216, 0.3267],
        [0.3766, 0.2888, 0.3346],
        [0.3419, 0.3116, 0.3465],
        [0.3461, 0.2898, 0.3641],
        [0.3625, 0.3012, 0.3364],
        [0.3488, 0.2968, 0.3544],
        [0.3964, 0.2762, 0.3273],
        [0.3847, 0.2893, 0.3260],
        [0.3713, 0.2837, 0.3450],
        [0.3665, 0.3013, 0.3322],
        [0.3880, 0.2858, 0.3262],
        [0.3848, 0.3012, 0.3140],
        [0.3450, 0.2791, 0.3759],
        [0.4106, 0.2584, 0.3310]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3570, 0.3039, 0.3391],
        [0.2992, 0.3159, 0.3850],
        [0.3583, 0.3178, 0.3239],
        [0.3322, 0.3286, 0.3392],
        [0.3605, 0.3227, 0.3167],
        [0.3545, 0.3137, 0.3318],
        [0.3151, 0.3377, 0.3472],
        [0.3309, 0.3236, 0.3456],
        [0.3532, 0.2981, 0.3486],
        [0.3338, 0.3186, 0.3475],
        [0.3238, 0.3215, 0.3547],
        [0.3311, 0.3239, 0.3450],
        [0.3294, 0.3277, 0.3430],
        [0.3376, 0.3244, 0.3380]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 13 [   0/390]  Loss: 0.4897 (0.490)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 2.121e-02
Train: 13 [  50/390]  Loss: 0.3400 (0.474)  Acc@1: 92.1875 (83.1189)  Acc@5: 100.0000 (99.2034)LR: 2.121e-02
Train: 13 [ 100/390]  Loss: 0.3303 (0.462)  Acc@1: 89.0625 (83.6170)  Acc@5: 100.0000 (99.2729)LR: 2.121e-02
Train: 13 [ 150/390]  Loss: 0.3851 (0.469)  Acc@1: 89.0625 (83.7955)  Acc@5: 98.4375 (99.3274)LR: 2.121e-02
Train: 13 [ 200/390]  Loss: 0.5722 (0.477)  Acc@1: 82.8125 (83.5044)  Acc@5: 98.4375 (99.3237)LR: 2.121e-02
Train: 13 [ 250/390]  Loss: 0.2874 (0.483)  Acc@1: 92.1875 (83.2234)  Acc@5: 100.0000 (99.3028)LR: 2.121e-02
Train: 13 [ 300/390]  Loss: 0.4759 (0.490)  Acc@1: 82.8125 (83.0565)  Acc@5: 100.0000 (99.2784)LR: 2.121e-02
Train: 13 [ 350/390]  Loss: 0.4827 (0.496)  Acc@1: 85.9375 (82.8659)  Acc@5: 100.0000 (99.2610)LR: 2.121e-02
Train: 13 [ 390/390]  Loss: 0.3965 (0.495)  Acc@1: 87.5000 (82.8720)  Acc@5: 100.0000 (99.2600)LR: 2.121e-02
train_acc 82.872000
Valid: 13 [   0/390]  Loss: 0.4528 (0.453)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 13 [  50/390]  Loss: 0.4570 (0.559)  Acc@1: 82.8125 (79.8100)  Acc@5: 100.0000 (99.2341)
Valid: 13 [ 100/390]  Loss: 0.6109 (0.556)  Acc@1: 71.8750 (80.1516)  Acc@5: 100.0000 (99.1491)
Valid: 13 [ 150/390]  Loss: 0.5066 (0.557)  Acc@1: 76.5625 (80.4325)  Acc@5: 100.0000 (99.0584)
Valid: 13 [ 200/390]  Loss: 0.6112 (0.555)  Acc@1: 71.8750 (80.3716)  Acc@5: 100.0000 (99.0361)
Valid: 13 [ 250/390]  Loss: 0.7434 (0.563)  Acc@1: 70.3125 (80.0984)  Acc@5: 100.0000 (99.0538)
Valid: 13 [ 300/390]  Loss: 0.4593 (0.562)  Acc@1: 84.3750 (80.1287)  Acc@5: 100.0000 (99.0345)
Valid: 13 [ 350/390]  Loss: 0.4443 (0.560)  Acc@1: 87.5000 (80.1994)  Acc@5: 100.0000 (99.0251)
Valid: 13 [ 390/390]  Loss: 0.7505 (0.559)  Acc@1: 77.5000 (80.1880)  Acc@5: 97.5000 (99.0720)
valid_acc 80.188000
epoch = 13   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3500, 0.3230, 0.3270],
        [0.3839, 0.2838, 0.3323],
        [0.3423, 0.3126, 0.3451],
        [0.3494, 0.2868, 0.3638],
        [0.3689, 0.2995, 0.3317],
        [0.3467, 0.2930, 0.3603],
        [0.4042, 0.2741, 0.3218],
        [0.3915, 0.2871, 0.3214],
        [0.3807, 0.2811, 0.3383],
        [0.3684, 0.3006, 0.3310],
        [0.3945, 0.2853, 0.3202],
        [0.3888, 0.3003, 0.3109],
        [0.3540, 0.2776, 0.3684],
        [0.4218, 0.2514, 0.3268]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3571, 0.3062, 0.3367],
        [0.2976, 0.3159, 0.3865],
        [0.3574, 0.3142, 0.3283],
        [0.3290, 0.3288, 0.3423],
        [0.3633, 0.3238, 0.3129],
        [0.3536, 0.3173, 0.3292],
        [0.3131, 0.3425, 0.3444],
        [0.3345, 0.3212, 0.3443],
        [0.3576, 0.2979, 0.3445],
        [0.3327, 0.3183, 0.3490],
        [0.3256, 0.3212, 0.3532],
        [0.3322, 0.3233, 0.3445],
        [0.3281, 0.3285, 0.3434],
        [0.3408, 0.3234, 0.3358]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 14 [   0/390]  Loss: 0.4991 (0.499)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)LR: 2.065e-02
Train: 14 [  50/390]  Loss: 0.3842 (0.492)  Acc@1: 84.3750 (83.1495)  Acc@5: 100.0000 (99.2034)LR: 2.065e-02
Train: 14 [ 100/390]  Loss: 0.3125 (0.480)  Acc@1: 87.5000 (83.1993)  Acc@5: 100.0000 (99.1801)LR: 2.065e-02
Train: 14 [ 150/390]  Loss: 0.3915 (0.487)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (99.2032)LR: 2.065e-02
Train: 14 [ 200/390]  Loss: 0.2443 (0.482)  Acc@1: 90.6250 (83.0613)  Acc@5: 100.0000 (99.2537)LR: 2.065e-02
Train: 14 [ 250/390]  Loss: 0.4706 (0.478)  Acc@1: 82.8125 (83.3105)  Acc@5: 98.4375 (99.2468)LR: 2.065e-02
Train: 14 [ 300/390]  Loss: 0.3972 (0.475)  Acc@1: 84.3750 (83.5496)  Acc@5: 100.0000 (99.2733)LR: 2.065e-02
Train: 14 [ 350/390]  Loss: 0.4352 (0.479)  Acc@1: 82.8125 (83.3511)  Acc@5: 100.0000 (99.2254)LR: 2.065e-02
Train: 14 [ 390/390]  Loss: 0.5238 (0.479)  Acc@1: 80.0000 (83.4080)  Acc@5: 100.0000 (99.2120)LR: 2.065e-02
train_acc 83.408000
Valid: 14 [   0/390]  Loss: 0.4884 (0.488)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 14 [  50/390]  Loss: 0.5874 (0.561)  Acc@1: 79.6875 (80.8824)  Acc@5: 98.4375 (99.4179)
Valid: 14 [ 100/390]  Loss: 0.6060 (0.543)  Acc@1: 78.1250 (81.4511)  Acc@5: 100.0000 (99.3193)
Valid: 14 [ 150/390]  Loss: 0.4856 (0.533)  Acc@1: 84.3750 (81.6950)  Acc@5: 98.4375 (99.2446)
Valid: 14 [ 200/390]  Loss: 0.5481 (0.529)  Acc@1: 78.1250 (81.8330)  Acc@5: 96.8750 (99.1604)
Valid: 14 [ 250/390]  Loss: 0.3505 (0.528)  Acc@1: 90.6250 (81.8103)  Acc@5: 100.0000 (99.1534)
Valid: 14 [ 300/390]  Loss: 0.4488 (0.527)  Acc@1: 81.2500 (81.8210)  Acc@5: 100.0000 (99.1227)
Valid: 14 [ 350/390]  Loss: 0.6201 (0.521)  Acc@1: 78.1250 (82.0201)  Acc@5: 98.4375 (99.1275)
Valid: 14 [ 390/390]  Loss: 0.6912 (0.521)  Acc@1: 80.0000 (82.0600)  Acc@5: 97.5000 (99.1360)
valid_acc 82.060000
epoch = 14   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3526, 0.3236, 0.3238],
        [0.3871, 0.2824, 0.3305],
        [0.3434, 0.3114, 0.3452],
        [0.3517, 0.2858, 0.3625],
        [0.3748, 0.2961, 0.3291],
        [0.3474, 0.2907, 0.3619],
        [0.4095, 0.2732, 0.3173],
        [0.3969, 0.2829, 0.3202],
        [0.3878, 0.2788, 0.3335],
        [0.3711, 0.2982, 0.3306],
        [0.3964, 0.2848, 0.3188],
        [0.3967, 0.2975, 0.3057],
        [0.3648, 0.2756, 0.3596],
        [0.4321, 0.2467, 0.3212]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3592, 0.3064, 0.3344],
        [0.2951, 0.3181, 0.3868],
        [0.3604, 0.3139, 0.3257],
        [0.3266, 0.3303, 0.3432],
        [0.3646, 0.3252, 0.3102],
        [0.3531, 0.3197, 0.3272],
        [0.3103, 0.3450, 0.3447],
        [0.3360, 0.3199, 0.3441],
        [0.3608, 0.2945, 0.3447],
        [0.3334, 0.3193, 0.3473],
        [0.3220, 0.3231, 0.3549],
        [0.3326, 0.3221, 0.3454],
        [0.3308, 0.3291, 0.3401],
        [0.3422, 0.3246, 0.3332]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 15 [   0/390]  Loss: 0.5480 (0.548)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)LR: 2.005e-02
Train: 15 [  50/390]  Loss: 0.5497 (0.450)  Acc@1: 84.3750 (83.6397)  Acc@5: 98.4375 (99.6936)LR: 2.005e-02
Train: 15 [ 100/390]  Loss: 0.9308 (0.445)  Acc@1: 73.4375 (84.2203)  Acc@5: 96.8750 (99.5514)LR: 2.005e-02
Train: 15 [ 150/390]  Loss: 0.6249 (0.450)  Acc@1: 79.6875 (84.0335)  Acc@5: 100.0000 (99.5550)LR: 2.005e-02
Train: 15 [ 200/390]  Loss: 0.3654 (0.459)  Acc@1: 89.0625 (83.6287)  Acc@5: 100.0000 (99.5180)LR: 2.005e-02
Train: 15 [ 250/390]  Loss: 0.4249 (0.463)  Acc@1: 82.8125 (83.5906)  Acc@5: 100.0000 (99.4895)LR: 2.005e-02
Train: 15 [ 300/390]  Loss: 0.7514 (0.461)  Acc@1: 78.1250 (83.7105)  Acc@5: 98.4375 (99.4757)LR: 2.005e-02
Train: 15 [ 350/390]  Loss: 0.6673 (0.466)  Acc@1: 82.8125 (83.6494)  Acc@5: 95.3125 (99.4168)LR: 2.005e-02
Train: 15 [ 390/390]  Loss: 0.4266 (0.465)  Acc@1: 82.5000 (83.6680)  Acc@5: 97.5000 (99.3960)LR: 2.005e-02
train_acc 83.668000
Valid: 15 [   0/390]  Loss: 0.6545 (0.654)  Acc@1: 78.1250 (78.1250)  Acc@5: 98.4375 (98.4375)
Valid: 15 [  50/390]  Loss: 0.5966 (0.550)  Acc@1: 78.1250 (80.5147)  Acc@5: 100.0000 (99.2034)
Valid: 15 [ 100/390]  Loss: 0.7023 (0.568)  Acc@1: 78.1250 (80.2908)  Acc@5: 95.3125 (99.1027)
Valid: 15 [ 150/390]  Loss: 0.6385 (0.574)  Acc@1: 79.6875 (79.8634)  Acc@5: 98.4375 (99.1411)
Valid: 15 [ 200/390]  Loss: 0.4450 (0.574)  Acc@1: 87.5000 (80.0140)  Acc@5: 98.4375 (99.0749)
Valid: 15 [ 250/390]  Loss: 0.4578 (0.575)  Acc@1: 79.6875 (80.0050)  Acc@5: 100.0000 (99.0413)
Valid: 15 [ 300/390]  Loss: 0.5296 (0.571)  Acc@1: 84.3750 (80.1184)  Acc@5: 98.4375 (99.0760)
Valid: 15 [ 350/390]  Loss: 0.7808 (0.575)  Acc@1: 75.0000 (79.9679)  Acc@5: 100.0000 (99.0785)
Valid: 15 [ 390/390]  Loss: 0.6442 (0.574)  Acc@1: 70.0000 (80.0000)  Acc@5: 97.5000 (99.0640)
valid_acc 80.000000
epoch = 15   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('skip_connect', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3537, 0.3230, 0.3233],
        [0.3921, 0.2766, 0.3313],
        [0.3429, 0.3116, 0.3455],
        [0.3559, 0.2839, 0.3603],
        [0.3826, 0.2945, 0.3229],
        [0.3473, 0.2876, 0.3651],
        [0.4164, 0.2680, 0.3156],
        [0.4042, 0.2791, 0.3167],
        [0.3984, 0.2741, 0.3275],
        [0.3730, 0.2971, 0.3299],
        [0.4059, 0.2819, 0.3123],
        [0.4062, 0.2952, 0.2986],
        [0.3742, 0.2715, 0.3543],
        [0.4445, 0.2389, 0.3166]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3623, 0.3037, 0.3340],
        [0.2909, 0.3224, 0.3866],
        [0.3628, 0.3103, 0.3269],
        [0.3209, 0.3322, 0.3470],
        [0.3653, 0.3273, 0.3073],
        [0.3568, 0.3192, 0.3240],
        [0.3056, 0.3467, 0.3476],
        [0.3381, 0.3207, 0.3412],
        [0.3594, 0.2924, 0.3482],
        [0.3388, 0.3188, 0.3425],
        [0.3210, 0.3264, 0.3526],
        [0.3340, 0.3221, 0.3439],
        [0.3306, 0.3327, 0.3367],
        [0.3412, 0.3260, 0.3327]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 16 [   0/390]  Loss: 0.3319 (0.332)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.943e-02
Train: 16 [  50/390]  Loss: 0.4514 (0.445)  Acc@1: 84.3750 (84.9877)  Acc@5: 100.0000 (99.2341)LR: 1.943e-02
Train: 16 [ 100/390]  Loss: 0.7468 (0.447)  Acc@1: 76.5625 (84.8236)  Acc@5: 96.8750 (99.2884)LR: 1.943e-02
Train: 16 [ 150/390]  Loss: 0.3499 (0.447)  Acc@1: 90.6250 (84.7993)  Acc@5: 100.0000 (99.3171)LR: 1.943e-02
Train: 16 [ 200/390]  Loss: 0.4451 (0.449)  Acc@1: 82.8125 (84.6549)  Acc@5: 100.0000 (99.3392)LR: 1.943e-02
Train: 16 [ 250/390]  Loss: 0.7021 (0.446)  Acc@1: 81.2500 (84.7049)  Acc@5: 100.0000 (99.3650)LR: 1.943e-02
Train: 16 [ 300/390]  Loss: 0.5123 (0.448)  Acc@1: 82.8125 (84.7280)  Acc@5: 100.0000 (99.3615)LR: 1.943e-02
Train: 16 [ 350/390]  Loss: 0.3391 (0.448)  Acc@1: 85.9375 (84.6911)  Acc@5: 100.0000 (99.3679)LR: 1.943e-02
Train: 16 [ 390/390]  Loss: 0.3704 (0.445)  Acc@1: 87.5000 (84.7560)  Acc@5: 100.0000 (99.4000)LR: 1.943e-02
train_acc 84.756000
Valid: 16 [   0/390]  Loss: 0.5261 (0.526)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 16 [  50/390]  Loss: 0.6071 (0.532)  Acc@1: 76.5625 (81.7402)  Acc@5: 100.0000 (99.1115)
Valid: 16 [ 100/390]  Loss: 0.5240 (0.532)  Acc@1: 81.2500 (81.4821)  Acc@5: 100.0000 (99.0563)
Valid: 16 [ 150/390]  Loss: 0.4615 (0.538)  Acc@1: 84.3750 (81.2397)  Acc@5: 98.4375 (99.0998)
Valid: 16 [ 200/390]  Loss: 0.4281 (0.532)  Acc@1: 81.2500 (81.3822)  Acc@5: 100.0000 (99.1682)
Valid: 16 [ 250/390]  Loss: 0.5712 (0.534)  Acc@1: 78.1250 (81.3496)  Acc@5: 100.0000 (99.1970)
Valid: 16 [ 300/390]  Loss: 0.5763 (0.533)  Acc@1: 78.1250 (81.4265)  Acc@5: 100.0000 (99.2162)
Valid: 16 [ 350/390]  Loss: 0.6236 (0.536)  Acc@1: 78.1250 (81.2144)  Acc@5: 100.0000 (99.1765)
Valid: 16 [ 390/390]  Loss: 0.3116 (0.537)  Acc@1: 90.0000 (81.1520)  Acc@5: 100.0000 (99.1760)
valid_acc 81.152000
epoch = 16   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3543, 0.3242, 0.3215],
        [0.3972, 0.2722, 0.3307],
        [0.3437, 0.3141, 0.3422],
        [0.3614, 0.2818, 0.3568],
        [0.3883, 0.2935, 0.3182],
        [0.3483, 0.2885, 0.3632],
        [0.4222, 0.2656, 0.3122],
        [0.4071, 0.2767, 0.3162],
        [0.4054, 0.2708, 0.3238],
        [0.3754, 0.2958, 0.3288],
        [0.4120, 0.2786, 0.3093],
        [0.4147, 0.2940, 0.2914],
        [0.3830, 0.2666, 0.3503],
        [0.4541, 0.2318, 0.3141]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3621, 0.3020, 0.3359],
        [0.2909, 0.3199, 0.3892],
        [0.3610, 0.3105, 0.3285],
        [0.3231, 0.3308, 0.3461],
        [0.3663, 0.3255, 0.3082],
        [0.3545, 0.3216, 0.3240],
        [0.3055, 0.3456, 0.3489],
        [0.3407, 0.3195, 0.3398],
        [0.3603, 0.2918, 0.3478],
        [0.3364, 0.3213, 0.3423],
        [0.3230, 0.3254, 0.3516],
        [0.3335, 0.3203, 0.3462],
        [0.3311, 0.3370, 0.3320],
        [0.3411, 0.3277, 0.3312]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 17 [   0/390]  Loss: 0.4479 (0.448)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)LR: 1.878e-02
Train: 17 [  50/390]  Loss: 0.3843 (0.429)  Acc@1: 85.9375 (85.0184)  Acc@5: 98.4375 (99.4179)LR: 1.878e-02
Train: 17 [ 100/390]  Loss: 0.2148 (0.432)  Acc@1: 93.7500 (85.0402)  Acc@5: 98.4375 (99.3967)LR: 1.878e-02
Train: 17 [ 150/390]  Loss: 0.5951 (0.440)  Acc@1: 79.6875 (84.7786)  Acc@5: 100.0000 (99.4102)LR: 1.878e-02
Train: 17 [ 200/390]  Loss: 0.5971 (0.443)  Acc@1: 76.5625 (84.6937)  Acc@5: 98.4375 (99.4092)LR: 1.878e-02
Train: 17 [ 250/390]  Loss: 0.3521 (0.445)  Acc@1: 84.3750 (84.6053)  Acc@5: 100.0000 (99.4460)LR: 1.878e-02
Train: 17 [ 300/390]  Loss: 0.6098 (0.452)  Acc@1: 78.1250 (84.2919)  Acc@5: 100.0000 (99.4342)LR: 1.878e-02
Train: 17 [ 350/390]  Loss: 0.5050 (0.451)  Acc@1: 81.2500 (84.2370)  Acc@5: 100.0000 (99.4525)LR: 1.878e-02
Train: 17 [ 390/390]  Loss: 0.3780 (0.448)  Acc@1: 85.0000 (84.3480)  Acc@5: 100.0000 (99.4680)LR: 1.878e-02
train_acc 84.348000
Valid: 17 [   0/390]  Loss: 0.4400 (0.440)  Acc@1: 84.3750 (84.3750)  Acc@5: 95.3125 (95.3125)
Valid: 17 [  50/390]  Loss: 0.4181 (0.511)  Acc@1: 87.5000 (82.8738)  Acc@5: 98.4375 (98.5600)
Valid: 17 [ 100/390]  Loss: 0.6846 (0.503)  Acc@1: 84.3750 (82.8125)  Acc@5: 95.3125 (98.8861)
Valid: 17 [ 150/390]  Loss: 0.5903 (0.509)  Acc@1: 78.1250 (82.9470)  Acc@5: 98.4375 (98.8307)
Valid: 17 [ 200/390]  Loss: 0.5096 (0.511)  Acc@1: 79.6875 (82.8047)  Acc@5: 98.4375 (98.8884)
Valid: 17 [ 250/390]  Loss: 0.7262 (0.515)  Acc@1: 75.0000 (82.6320)  Acc@5: 95.3125 (98.7923)
Valid: 17 [ 300/390]  Loss: 0.3788 (0.516)  Acc@1: 87.5000 (82.6516)  Acc@5: 100.0000 (98.8632)
Valid: 17 [ 350/390]  Loss: 0.4741 (0.512)  Acc@1: 85.9375 (82.6300)  Acc@5: 98.4375 (98.8871)
Valid: 17 [ 390/390]  Loss: 0.3311 (0.516)  Acc@1: 90.0000 (82.5360)  Acc@5: 100.0000 (98.8640)
valid_acc 82.536000
epoch = 17   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3528, 0.3241, 0.3231],
        [0.4028, 0.2664, 0.3308],
        [0.3437, 0.3141, 0.3421],
        [0.3642, 0.2806, 0.3552],
        [0.3946, 0.2945, 0.3109],
        [0.3520, 0.2902, 0.3578],
        [0.4278, 0.2652, 0.3070],
        [0.4153, 0.2772, 0.3076],
        [0.4146, 0.2695, 0.3160],
        [0.3822, 0.2958, 0.3220],
        [0.4175, 0.2773, 0.3051],
        [0.4225, 0.2952, 0.2823],
        [0.3900, 0.2654, 0.3446],
        [0.4682, 0.2246, 0.3071]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3648, 0.2992, 0.3360],
        [0.2875, 0.3212, 0.3913],
        [0.3622, 0.3100, 0.3277],
        [0.3207, 0.3336, 0.3457],
        [0.3670, 0.3257, 0.3072],
        [0.3535, 0.3239, 0.3226],
        [0.3056, 0.3491, 0.3453],
        [0.3434, 0.3193, 0.3373],
        [0.3599, 0.2923, 0.3478],
        [0.3358, 0.3229, 0.3414],
        [0.3219, 0.3242, 0.3539],
        [0.3349, 0.3231, 0.3420],
        [0.3293, 0.3400, 0.3307],
        [0.3433, 0.3276, 0.3291]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 18 [   0/390]  Loss: 0.3181 (0.318)  Acc@1: 90.6250 (90.6250)  Acc@5: 98.4375 (98.4375)LR: 1.811e-02
Train: 18 [  50/390]  Loss: 0.4902 (0.422)  Acc@1: 78.1250 (85.6311)  Acc@5: 100.0000 (99.2341)LR: 1.811e-02
Train: 18 [ 100/390]  Loss: 0.8243 (0.421)  Acc@1: 78.1250 (85.5353)  Acc@5: 100.0000 (99.4431)LR: 1.811e-02
Train: 18 [ 150/390]  Loss: 0.3228 (0.425)  Acc@1: 87.5000 (85.2649)  Acc@5: 100.0000 (99.4412)LR: 1.811e-02
Train: 18 [ 200/390]  Loss: 0.5024 (0.428)  Acc@1: 84.3750 (85.2146)  Acc@5: 98.4375 (99.4325)LR: 1.811e-02
Train: 18 [ 250/390]  Loss: 0.5182 (0.434)  Acc@1: 84.3750 (85.0847)  Acc@5: 100.0000 (99.4397)LR: 1.811e-02
Train: 18 [ 300/390]  Loss: 0.3129 (0.435)  Acc@1: 90.6250 (85.0291)  Acc@5: 100.0000 (99.4134)LR: 1.811e-02
Train: 18 [ 350/390]  Loss: 0.3014 (0.439)  Acc@1: 89.0625 (84.9448)  Acc@5: 100.0000 (99.3812)LR: 1.811e-02
Train: 18 [ 390/390]  Loss: 0.4437 (0.437)  Acc@1: 85.0000 (84.9720)  Acc@5: 100.0000 (99.3960)LR: 1.811e-02
train_acc 84.972000
Valid: 18 [   0/390]  Loss: 0.3266 (0.327)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)
Valid: 18 [  50/390]  Loss: 0.4617 (0.530)  Acc@1: 85.9375 (81.0968)  Acc@5: 98.4375 (99.0809)
Valid: 18 [ 100/390]  Loss: 0.5636 (0.519)  Acc@1: 84.3750 (81.4047)  Acc@5: 98.4375 (99.1646)
Valid: 18 [ 150/390]  Loss: 0.5811 (0.524)  Acc@1: 85.9375 (81.4776)  Acc@5: 98.4375 (99.1101)
Valid: 18 [ 200/390]  Loss: 0.3336 (0.518)  Acc@1: 89.0625 (81.8175)  Acc@5: 100.0000 (99.2226)
Valid: 18 [ 250/390]  Loss: 0.4846 (0.519)  Acc@1: 85.9375 (81.7854)  Acc@5: 100.0000 (99.2530)
Valid: 18 [ 300/390]  Loss: 0.5387 (0.520)  Acc@1: 81.2500 (81.7899)  Acc@5: 96.8750 (99.2369)
Valid: 18 [ 350/390]  Loss: 0.4441 (0.520)  Acc@1: 82.8125 (81.9311)  Acc@5: 98.4375 (99.2299)
Valid: 18 [ 390/390]  Loss: 0.6602 (0.524)  Acc@1: 82.5000 (81.7600)  Acc@5: 100.0000 (99.2160)
valid_acc 81.760000
epoch = 18   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('skip_connect', 2), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3526, 0.3222, 0.3252],
        [0.4078, 0.2634, 0.3288],
        [0.3449, 0.3138, 0.3414],
        [0.3718, 0.2788, 0.3494],
        [0.3979, 0.2940, 0.3081],
        [0.3574, 0.2894, 0.3532],
        [0.4361, 0.2616, 0.3023],
        [0.4216, 0.2728, 0.3056],
        [0.4268, 0.2639, 0.3093],
        [0.3864, 0.2958, 0.3178],
        [0.4244, 0.2754, 0.3002],
        [0.4294, 0.2943, 0.2763],
        [0.3977, 0.2613, 0.3410],
        [0.4792, 0.2181, 0.3027]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3639, 0.2980, 0.3381],
        [0.2858, 0.3225, 0.3918],
        [0.3651, 0.3105, 0.3244],
        [0.3192, 0.3371, 0.3437],
        [0.3692, 0.3253, 0.3056],
        [0.3582, 0.3233, 0.3185],
        [0.3071, 0.3502, 0.3427],
        [0.3411, 0.3183, 0.3406],
        [0.3622, 0.2896, 0.3482],
        [0.3342, 0.3244, 0.3415],
        [0.3191, 0.3288, 0.3521],
        [0.3339, 0.3242, 0.3419],
        [0.3284, 0.3405, 0.3311],
        [0.3464, 0.3255, 0.3281]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 19 [   0/390]  Loss: 0.3057 (0.306)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.742e-02
Train: 19 [  50/390]  Loss: 0.3249 (0.428)  Acc@1: 84.3750 (84.5895)  Acc@5: 100.0000 (99.5404)LR: 1.742e-02
Train: 19 [ 100/390]  Loss: 0.4626 (0.429)  Acc@1: 82.8125 (84.7772)  Acc@5: 100.0000 (99.4431)LR: 1.742e-02
Train: 19 [ 150/390]  Loss: 0.5133 (0.424)  Acc@1: 79.6875 (85.0062)  Acc@5: 98.4375 (99.4826)LR: 1.742e-02
Train: 19 [ 200/390]  Loss: 0.3069 (0.425)  Acc@1: 85.9375 (84.9036)  Acc@5: 100.0000 (99.5258)LR: 1.742e-02
Train: 19 [ 250/390]  Loss: 0.3268 (0.429)  Acc@1: 89.0625 (84.7983)  Acc@5: 100.0000 (99.5082)LR: 1.742e-02
Train: 19 [ 300/390]  Loss: 0.3423 (0.426)  Acc@1: 90.6250 (84.9356)  Acc@5: 98.4375 (99.5276)LR: 1.742e-02
Train: 19 [ 350/390]  Loss: 0.4668 (0.427)  Acc@1: 87.5000 (84.9136)  Acc@5: 98.4375 (99.4747)LR: 1.742e-02
Train: 19 [ 390/390]  Loss: 0.5634 (0.430)  Acc@1: 80.0000 (84.8120)  Acc@5: 100.0000 (99.4600)LR: 1.742e-02
train_acc 84.812000
Valid: 19 [   0/390]  Loss: 0.2213 (0.221)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)
Valid: 19 [  50/390]  Loss: 0.4122 (0.474)  Acc@1: 87.5000 (83.7010)  Acc@5: 98.4375 (99.2034)
Valid: 19 [ 100/390]  Loss: 0.5130 (0.480)  Acc@1: 81.2500 (83.5241)  Acc@5: 100.0000 (99.2729)
Valid: 19 [ 150/390]  Loss: 0.3975 (0.481)  Acc@1: 90.6250 (83.5989)  Acc@5: 100.0000 (99.2343)
Valid: 19 [ 200/390]  Loss: 0.3530 (0.480)  Acc@1: 92.1875 (83.8464)  Acc@5: 100.0000 (99.2304)
Valid: 19 [ 250/390]  Loss: 0.4491 (0.477)  Acc@1: 84.3750 (83.9268)  Acc@5: 100.0000 (99.2405)
Valid: 19 [ 300/390]  Loss: 0.2665 (0.476)  Acc@1: 93.7500 (83.8819)  Acc@5: 100.0000 (99.2629)
Valid: 19 [ 350/390]  Loss: 0.4707 (0.481)  Acc@1: 85.9375 (83.6494)  Acc@5: 98.4375 (99.2566)
Valid: 19 [ 390/390]  Loss: 0.5432 (0.486)  Acc@1: 77.5000 (83.4800)  Acc@5: 100.0000 (99.2000)
valid_acc 83.480000
epoch = 19   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3540, 0.3219, 0.3241],
        [0.4146, 0.2586, 0.3267],
        [0.3446, 0.3141, 0.3413],
        [0.3774, 0.2741, 0.3485],
        [0.4040, 0.2912, 0.3048],
        [0.3554, 0.2863, 0.3583],
        [0.4456, 0.2562, 0.2982],
        [0.4283, 0.2700, 0.3017],
        [0.4341, 0.2597, 0.3063],
        [0.3891, 0.2953, 0.3156],
        [0.4322, 0.2716, 0.2962],
        [0.4340, 0.2913, 0.2748],
        [0.4018, 0.2584, 0.3399],
        [0.4897, 0.2141, 0.2962]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3683, 0.2955, 0.3361],
        [0.2835, 0.3224, 0.3941],
        [0.3670, 0.3099, 0.3231],
        [0.3170, 0.3378, 0.3451],
        [0.3707, 0.3230, 0.3063],
        [0.3608, 0.3231, 0.3161],
        [0.3050, 0.3495, 0.3455],
        [0.3433, 0.3150, 0.3417],
        [0.3649, 0.2876, 0.3475],
        [0.3339, 0.3228, 0.3433],
        [0.3161, 0.3299, 0.3540],
        [0.3322, 0.3245, 0.3433],
        [0.3278, 0.3449, 0.3273],
        [0.3473, 0.3244, 0.3284]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 20 [   0/390]  Loss: 0.3965 (0.397)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.671e-02
Train: 20 [  50/390]  Loss: 0.3131 (0.355)  Acc@1: 90.6250 (87.3162)  Acc@5: 100.0000 (99.6324)LR: 1.671e-02
Train: 20 [ 100/390]  Loss: 0.4651 (0.380)  Acc@1: 82.8125 (86.4944)  Acc@5: 100.0000 (99.7061)LR: 1.671e-02
Train: 20 [ 150/390]  Loss: 0.2571 (0.394)  Acc@1: 92.1875 (86.1548)  Acc@5: 100.0000 (99.6896)LR: 1.671e-02
Train: 20 [ 200/390]  Loss: 0.3890 (0.395)  Acc@1: 85.9375 (86.1318)  Acc@5: 100.0000 (99.6424)LR: 1.671e-02
Train: 20 [ 250/390]  Loss: 0.4074 (0.402)  Acc@1: 89.0625 (86.0745)  Acc@5: 100.0000 (99.5954)LR: 1.671e-02
Train: 20 [ 300/390]  Loss: 0.4050 (0.405)  Acc@1: 84.3750 (85.9271)  Acc@5: 100.0000 (99.5691)LR: 1.671e-02
Train: 20 [ 350/390]  Loss: 0.5295 (0.405)  Acc@1: 85.9375 (86.0399)  Acc@5: 100.0000 (99.5281)LR: 1.671e-02
Train: 20 [ 390/390]  Loss: 0.3493 (0.405)  Acc@1: 85.0000 (85.9680)  Acc@5: 100.0000 (99.5400)LR: 1.671e-02
train_acc 85.968000
Valid: 20 [   0/390]  Loss: 0.3181 (0.318)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)
Valid: 20 [  50/390]  Loss: 0.3985 (0.484)  Acc@1: 84.3750 (82.9657)  Acc@5: 100.0000 (99.3260)
Valid: 20 [ 100/390]  Loss: 0.4289 (0.474)  Acc@1: 85.9375 (83.4158)  Acc@5: 100.0000 (99.3038)
Valid: 20 [ 150/390]  Loss: 0.5218 (0.484)  Acc@1: 84.3750 (82.9574)  Acc@5: 100.0000 (99.2653)
Valid: 20 [ 200/390]  Loss: 0.5137 (0.478)  Acc@1: 82.8125 (83.3722)  Acc@5: 100.0000 (99.2071)
Valid: 20 [ 250/390]  Loss: 0.3414 (0.470)  Acc@1: 89.0625 (83.6591)  Acc@5: 100.0000 (99.2530)
Valid: 20 [ 300/390]  Loss: 0.5403 (0.475)  Acc@1: 79.6875 (83.5444)  Acc@5: 100.0000 (99.2577)
Valid: 20 [ 350/390]  Loss: 0.3559 (0.476)  Acc@1: 89.0625 (83.6672)  Acc@5: 100.0000 (99.2610)
Valid: 20 [ 390/390]  Loss: 0.3017 (0.474)  Acc@1: 90.0000 (83.7240)  Acc@5: 100.0000 (99.2640)
valid_acc 83.724000
epoch = 20   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3547, 0.3236, 0.3217],
        [0.4196, 0.2564, 0.3240],
        [0.3439, 0.3151, 0.3411],
        [0.3806, 0.2740, 0.3454],
        [0.4120, 0.2907, 0.2973],
        [0.3570, 0.2863, 0.3567],
        [0.4579, 0.2534, 0.2887],
        [0.4363, 0.2677, 0.2960],
        [0.4444, 0.2559, 0.2997],
        [0.3885, 0.2973, 0.3142],
        [0.4383, 0.2705, 0.2913],
        [0.4390, 0.2897, 0.2713],
        [0.4108, 0.2569, 0.3323],
        [0.5016, 0.2088, 0.2896]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3678, 0.2937, 0.3385],
        [0.2837, 0.3191, 0.3973],
        [0.3650, 0.3097, 0.3253],
        [0.3159, 0.3412, 0.3429],
        [0.3749, 0.3224, 0.3027],
        [0.3588, 0.3258, 0.3154],
        [0.3047, 0.3491, 0.3462],
        [0.3455, 0.3151, 0.3394],
        [0.3675, 0.2874, 0.3451],
        [0.3330, 0.3249, 0.3421],
        [0.3133, 0.3313, 0.3554],
        [0.3346, 0.3238, 0.3416],
        [0.3283, 0.3467, 0.3250],
        [0.3502, 0.3237, 0.3262]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 21 [   0/390]  Loss: 0.3950 (0.395)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.598e-02
Train: 21 [  50/390]  Loss: 0.5133 (0.416)  Acc@1: 79.6875 (85.3248)  Acc@5: 100.0000 (99.4179)LR: 1.598e-02
Train: 21 [ 100/390]  Loss: 0.2234 (0.399)  Acc@1: 95.3125 (86.0767)  Acc@5: 100.0000 (99.4895)LR: 1.598e-02
Train: 21 [ 150/390]  Loss: 0.2866 (0.402)  Acc@1: 90.6250 (85.9582)  Acc@5: 98.4375 (99.4723)LR: 1.598e-02
Train: 21 [ 200/390]  Loss: 0.5736 (0.404)  Acc@1: 78.1250 (85.8831)  Acc@5: 100.0000 (99.4869)LR: 1.598e-02
Train: 21 [ 250/390]  Loss: 0.3045 (0.407)  Acc@1: 87.5000 (85.7819)  Acc@5: 100.0000 (99.4895)LR: 1.598e-02
Train: 21 [ 300/390]  Loss: 0.4816 (0.410)  Acc@1: 84.3750 (85.8181)  Acc@5: 100.0000 (99.4601)LR: 1.598e-02
Train: 21 [ 350/390]  Loss: 0.4953 (0.409)  Acc@1: 82.8125 (85.8440)  Acc@5: 100.0000 (99.4747)LR: 1.598e-02
Train: 21 [ 390/390]  Loss: 0.3685 (0.410)  Acc@1: 85.0000 (85.8160)  Acc@5: 100.0000 (99.4600)LR: 1.598e-02
train_acc 85.816000
Valid: 21 [   0/390]  Loss: 0.3631 (0.363)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 21 [  50/390]  Loss: 0.5760 (0.516)  Acc@1: 73.4375 (81.6789)  Acc@5: 96.8750 (99.0809)
Valid: 21 [ 100/390]  Loss: 0.3488 (0.515)  Acc@1: 87.5000 (82.1627)  Acc@5: 100.0000 (99.1955)
Valid: 21 [ 150/390]  Loss: 0.3559 (0.509)  Acc@1: 85.9375 (82.3675)  Acc@5: 100.0000 (99.2446)
Valid: 21 [ 200/390]  Loss: 0.4559 (0.506)  Acc@1: 89.0625 (82.5016)  Acc@5: 100.0000 (99.2615)
Valid: 21 [ 250/390]  Loss: 0.5045 (0.506)  Acc@1: 82.8125 (82.3518)  Acc@5: 96.8750 (99.2530)
Valid: 21 [ 300/390]  Loss: 0.6769 (0.509)  Acc@1: 79.6875 (82.2778)  Acc@5: 96.8750 (99.2421)
Valid: 21 [ 350/390]  Loss: 0.7007 (0.504)  Acc@1: 71.8750 (82.4475)  Acc@5: 96.8750 (99.2477)
Valid: 21 [ 390/390]  Loss: 0.2984 (0.508)  Acc@1: 87.5000 (82.4120)  Acc@5: 100.0000 (99.2440)
valid_acc 82.412000
epoch = 21   
 genotype = Genotype(normal=[('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3549, 0.3228, 0.3223],
        [0.4248, 0.2527, 0.3225],
        [0.3465, 0.3163, 0.3372],
        [0.3864, 0.2718, 0.3418],
        [0.4162, 0.2890, 0.2948],
        [0.3614, 0.2827, 0.3559],
        [0.4643, 0.2470, 0.2887],
        [0.4430, 0.2630, 0.2940],
        [0.4529, 0.2503, 0.2968],
        [0.3929, 0.2942, 0.3129],
        [0.4456, 0.2699, 0.2845],
        [0.4456, 0.2871, 0.2673],
        [0.4207, 0.2530, 0.3262],
        [0.5118, 0.2044, 0.2838]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3686, 0.2933, 0.3381],
        [0.2827, 0.3157, 0.4017],
        [0.3636, 0.3090, 0.3274],
        [0.3179, 0.3399, 0.3422],
        [0.3756, 0.3230, 0.3014],
        [0.3586, 0.3298, 0.3116],
        [0.3083, 0.3476, 0.3441],
        [0.3465, 0.3137, 0.3398],
        [0.3660, 0.2853, 0.3487],
        [0.3316, 0.3280, 0.3404],
        [0.3130, 0.3300, 0.3569],
        [0.3351, 0.3223, 0.3426],
        [0.3289, 0.3473, 0.3238],
        [0.3500, 0.3213, 0.3287]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 22 [   0/390]  Loss: 0.2421 (0.242)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.525e-02
Train: 22 [  50/390]  Loss: 0.3628 (0.382)  Acc@1: 85.9375 (87.2243)  Acc@5: 100.0000 (99.6017)LR: 1.525e-02
Train: 22 [ 100/390]  Loss: 0.2247 (0.382)  Acc@1: 90.6250 (87.0050)  Acc@5: 100.0000 (99.5668)LR: 1.525e-02
Train: 22 [ 150/390]  Loss: 0.2317 (0.383)  Acc@1: 93.7500 (86.8274)  Acc@5: 100.0000 (99.5240)LR: 1.525e-02
Train: 22 [ 200/390]  Loss: 0.4649 (0.384)  Acc@1: 79.6875 (86.6604)  Acc@5: 100.0000 (99.5647)LR: 1.525e-02
Train: 22 [ 250/390]  Loss: 0.4636 (0.391)  Acc@1: 87.5000 (86.4044)  Acc@5: 96.8750 (99.4895)LR: 1.525e-02
Train: 22 [ 300/390]  Loss: 0.3968 (0.396)  Acc@1: 85.9375 (86.2386)  Acc@5: 100.0000 (99.4601)LR: 1.525e-02
Train: 22 [ 350/390]  Loss: 0.3184 (0.396)  Acc@1: 85.9375 (86.3248)  Acc@5: 100.0000 (99.4925)LR: 1.525e-02
Train: 22 [ 390/390]  Loss: 0.6025 (0.396)  Acc@1: 85.0000 (86.3520)  Acc@5: 100.0000 (99.4920)LR: 1.525e-02
train_acc 86.352000
Valid: 22 [   0/390]  Loss: 0.5268 (0.527)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 22 [  50/390]  Loss: 0.3200 (0.482)  Acc@1: 85.9375 (83.2414)  Acc@5: 100.0000 (99.1728)
Valid: 22 [ 100/390]  Loss: 0.5306 (0.479)  Acc@1: 82.8125 (83.4468)  Acc@5: 100.0000 (99.1337)
Valid: 22 [ 150/390]  Loss: 0.4569 (0.485)  Acc@1: 82.8125 (83.4954)  Acc@5: 100.0000 (99.0480)
Valid: 22 [ 200/390]  Loss: 0.2904 (0.484)  Acc@1: 89.0625 (83.5743)  Acc@5: 100.0000 (99.1371)
Valid: 22 [ 250/390]  Loss: 0.4942 (0.485)  Acc@1: 81.2500 (83.5097)  Acc@5: 100.0000 (99.1409)
Valid: 22 [ 300/390]  Loss: 0.5574 (0.490)  Acc@1: 78.1250 (83.4043)  Acc@5: 98.4375 (99.0812)
Valid: 22 [ 350/390]  Loss: 0.5193 (0.490)  Acc@1: 81.2500 (83.3244)  Acc@5: 100.0000 (99.1097)
Valid: 22 [ 390/390]  Loss: 0.3938 (0.489)  Acc@1: 85.0000 (83.3800)  Acc@5: 100.0000 (99.0800)
valid_acc 83.380000
epoch = 22   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3560, 0.3242, 0.3198],
        [0.4285, 0.2468, 0.3247],
        [0.3462, 0.3154, 0.3384],
        [0.3947, 0.2696, 0.3357],
        [0.4232, 0.2885, 0.2883],
        [0.3633, 0.2814, 0.3553],
        [0.4754, 0.2409, 0.2837],
        [0.4477, 0.2595, 0.2927],
        [0.4665, 0.2446, 0.2890],
        [0.3942, 0.2930, 0.3128],
        [0.4536, 0.2672, 0.2792],
        [0.4501, 0.2863, 0.2637],
        [0.4337, 0.2500, 0.3163],
        [0.5232, 0.1997, 0.2771]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3747, 0.2915, 0.3338],
        [0.2790, 0.3172, 0.4038],
        [0.3664, 0.3091, 0.3245],
        [0.3163, 0.3400, 0.3437],
        [0.3793, 0.3211, 0.2996],
        [0.3581, 0.3304, 0.3115],
        [0.3059, 0.3472, 0.3469],
        [0.3484, 0.3129, 0.3386],
        [0.3669, 0.2842, 0.3489],
        [0.3300, 0.3311, 0.3389],
        [0.3094, 0.3314, 0.3593],
        [0.3358, 0.3206, 0.3436],
        [0.3309, 0.3484, 0.3207],
        [0.3517, 0.3203, 0.3279]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 23 [   0/390]  Loss: 0.6891 (0.689)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)LR: 1.450e-02
Train: 23 [  50/390]  Loss: 0.1775 (0.385)  Acc@1: 95.3125 (86.8260)  Acc@5: 100.0000 (99.6936)LR: 1.450e-02
Train: 23 [ 100/390]  Loss: 0.3216 (0.393)  Acc@1: 90.6250 (86.2160)  Acc@5: 98.4375 (99.5514)LR: 1.450e-02
Train: 23 [ 150/390]  Loss: 0.2963 (0.387)  Acc@1: 90.6250 (86.3204)  Acc@5: 98.4375 (99.5964)LR: 1.450e-02
Train: 23 [ 200/390]  Loss: 0.3163 (0.385)  Acc@1: 87.5000 (86.3728)  Acc@5: 100.0000 (99.5958)LR: 1.450e-02
Train: 23 [ 250/390]  Loss: 0.3684 (0.387)  Acc@1: 89.0625 (86.3110)  Acc@5: 100.0000 (99.5705)LR: 1.450e-02
Train: 23 [ 300/390]  Loss: 0.4100 (0.389)  Acc@1: 87.5000 (86.2853)  Acc@5: 100.0000 (99.5691)LR: 1.450e-02
Train: 23 [ 350/390]  Loss: 0.5368 (0.386)  Acc@1: 84.3750 (86.4583)  Acc@5: 100.0000 (99.5682)LR: 1.450e-02
Train: 23 [ 390/390]  Loss: 0.2360 (0.390)  Acc@1: 95.0000 (86.3720)  Acc@5: 100.0000 (99.5480)LR: 1.450e-02
train_acc 86.372000
Valid: 23 [   0/390]  Loss: 0.4573 (0.457)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 23 [  50/390]  Loss: 0.2604 (0.425)  Acc@1: 92.1875 (85.2328)  Acc@5: 100.0000 (99.3260)
Valid: 23 [ 100/390]  Loss: 0.5787 (0.435)  Acc@1: 76.5625 (84.9474)  Acc@5: 98.4375 (99.3038)
Valid: 23 [ 150/390]  Loss: 0.3574 (0.434)  Acc@1: 87.5000 (85.0166)  Acc@5: 100.0000 (99.3274)
Valid: 23 [ 200/390]  Loss: 0.4858 (0.439)  Acc@1: 81.2500 (84.8414)  Acc@5: 100.0000 (99.3392)
Valid: 23 [ 250/390]  Loss: 0.5308 (0.440)  Acc@1: 82.8125 (84.8481)  Acc@5: 98.4375 (99.3215)
Valid: 23 [ 300/390]  Loss: 0.4555 (0.440)  Acc@1: 81.2500 (84.8318)  Acc@5: 100.0000 (99.3355)
Valid: 23 [ 350/390]  Loss: 0.6935 (0.441)  Acc@1: 79.6875 (84.9047)  Acc@5: 96.8750 (99.3100)
Valid: 23 [ 390/390]  Loss: 0.5271 (0.445)  Acc@1: 85.0000 (84.7160)  Acc@5: 97.5000 (99.2960)
valid_acc 84.716000
epoch = 23   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3582, 0.3238, 0.3180],
        [0.4301, 0.2416, 0.3283],
        [0.3524, 0.3150, 0.3327],
        [0.3980, 0.2687, 0.3333],
        [0.4276, 0.2863, 0.2861],
        [0.3680, 0.2809, 0.3511],
        [0.4844, 0.2380, 0.2776],
        [0.4526, 0.2561, 0.2913],
        [0.4788, 0.2400, 0.2812],
        [0.4013, 0.2912, 0.3075],
        [0.4593, 0.2675, 0.2732],
        [0.4551, 0.2842, 0.2607],
        [0.4463, 0.2461, 0.3076],
        [0.5362, 0.1932, 0.2706]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3787, 0.2853, 0.3359],
        [0.2768, 0.3170, 0.4062],
        [0.3694, 0.3048, 0.3258],
        [0.3128, 0.3383, 0.3489],
        [0.3828, 0.3191, 0.2981],
        [0.3628, 0.3277, 0.3095],
        [0.3038, 0.3477, 0.3485],
        [0.3476, 0.3104, 0.3419],
        [0.3719, 0.2809, 0.3472],
        [0.3311, 0.3358, 0.3332],
        [0.3058, 0.3370, 0.3572],
        [0.3377, 0.3181, 0.3442],
        [0.3299, 0.3478, 0.3224],
        [0.3526, 0.3163, 0.3311]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 24 [   0/390]  Loss: 0.2314 (0.231)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.375e-02
Train: 24 [  50/390]  Loss: 0.3320 (0.352)  Acc@1: 87.5000 (87.4387)  Acc@5: 100.0000 (99.5711)LR: 1.375e-02
Train: 24 [ 100/390]  Loss: 0.4017 (0.364)  Acc@1: 79.6875 (87.2989)  Acc@5: 100.0000 (99.6132)LR: 1.375e-02
Train: 24 [ 150/390]  Loss: 0.3079 (0.360)  Acc@1: 90.6250 (87.6449)  Acc@5: 100.0000 (99.5964)LR: 1.375e-02
Train: 24 [ 200/390]  Loss: 0.3856 (0.361)  Acc@1: 84.3750 (87.6244)  Acc@5: 98.4375 (99.5958)LR: 1.375e-02
Train: 24 [ 250/390]  Loss: 0.3015 (0.363)  Acc@1: 90.6250 (87.4191)  Acc@5: 100.0000 (99.5705)LR: 1.375e-02
Train: 24 [ 300/390]  Loss: 0.2787 (0.365)  Acc@1: 93.7500 (87.3443)  Acc@5: 100.0000 (99.5484)LR: 1.375e-02
Train: 24 [ 350/390]  Loss: 0.3174 (0.370)  Acc@1: 89.0625 (87.1172)  Acc@5: 100.0000 (99.5192)LR: 1.375e-02
Train: 24 [ 390/390]  Loss: 0.4864 (0.377)  Acc@1: 82.5000 (86.9680)  Acc@5: 100.0000 (99.5120)LR: 1.375e-02
train_acc 86.968000
Valid: 24 [   0/390]  Loss: 0.3331 (0.333)  Acc@1: 90.6250 (90.6250)  Acc@5: 98.4375 (98.4375)
Valid: 24 [  50/390]  Loss: 0.4648 (0.488)  Acc@1: 84.3750 (83.4252)  Acc@5: 100.0000 (99.0502)
Valid: 24 [ 100/390]  Loss: 0.5663 (0.506)  Acc@1: 82.8125 (82.6114)  Acc@5: 98.4375 (99.0254)
Valid: 24 [ 150/390]  Loss: 0.4650 (0.506)  Acc@1: 82.8125 (82.6159)  Acc@5: 98.4375 (98.9859)
Valid: 24 [ 200/390]  Loss: 0.5019 (0.506)  Acc@1: 82.8125 (82.5171)  Acc@5: 100.0000 (98.9894)
Valid: 24 [ 250/390]  Loss: 0.6311 (0.512)  Acc@1: 79.6875 (82.2834)  Acc@5: 98.4375 (98.9791)
Valid: 24 [ 300/390]  Loss: 0.3827 (0.513)  Acc@1: 82.8125 (82.3349)  Acc@5: 100.0000 (98.9722)
Valid: 24 [ 350/390]  Loss: 0.3970 (0.513)  Acc@1: 89.0625 (82.4252)  Acc@5: 100.0000 (98.9717)
Valid: 24 [ 390/390]  Loss: 0.4750 (0.513)  Acc@1: 82.5000 (82.4480)  Acc@5: 100.0000 (98.9800)
valid_acc 82.448000
epoch = 24   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('skip_connect', 2), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3607, 0.3213, 0.3180],
        [0.4338, 0.2408, 0.3253],
        [0.3531, 0.3141, 0.3328],
        [0.4038, 0.2678, 0.3284],
        [0.4345, 0.2849, 0.2806],
        [0.3732, 0.2795, 0.3473],
        [0.4896, 0.2337, 0.2766],
        [0.4610, 0.2535, 0.2855],
        [0.4868, 0.2356, 0.2776],
        [0.4053, 0.2889, 0.3057],
        [0.4690, 0.2667, 0.2643],
        [0.4613, 0.2795, 0.2592],
        [0.4555, 0.2426, 0.3020],
        [0.5498, 0.1876, 0.2626]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3769, 0.2831, 0.3400],
        [0.2760, 0.3181, 0.4059],
        [0.3713, 0.3061, 0.3227],
        [0.3123, 0.3374, 0.3503],
        [0.3813, 0.3229, 0.2957],
        [0.3629, 0.3289, 0.3082],
        [0.3045, 0.3484, 0.3471],
        [0.3491, 0.3096, 0.3413],
        [0.3755, 0.2811, 0.3433],
        [0.3297, 0.3370, 0.3333],
        [0.3039, 0.3376, 0.3585],
        [0.3372, 0.3199, 0.3430],
        [0.3305, 0.3477, 0.3218],
        [0.3534, 0.3157, 0.3309]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 25 [   0/390]  Loss: 0.4034 (0.403)  Acc@1: 93.7500 (93.7500)  Acc@5: 98.4375 (98.4375)LR: 1.300e-02
Train: 25 [  50/390]  Loss: 0.4890 (0.367)  Acc@1: 82.8125 (87.4387)  Acc@5: 100.0000 (99.5711)LR: 1.300e-02
Train: 25 [ 100/390]  Loss: 0.5919 (0.382)  Acc@1: 84.3750 (86.6337)  Acc@5: 100.0000 (99.6442)LR: 1.300e-02
Train: 25 [ 150/390]  Loss: 0.4329 (0.380)  Acc@1: 85.9375 (86.6825)  Acc@5: 100.0000 (99.6482)LR: 1.300e-02
Train: 25 [ 200/390]  Loss: 0.3313 (0.374)  Acc@1: 87.5000 (86.9170)  Acc@5: 100.0000 (99.6891)LR: 1.300e-02
Train: 25 [ 250/390]  Loss: 0.2368 (0.377)  Acc@1: 89.0625 (86.8588)  Acc@5: 100.0000 (99.6327)LR: 1.300e-02
Train: 25 [ 300/390]  Loss: 0.5075 (0.377)  Acc@1: 82.8125 (86.8615)  Acc@5: 100.0000 (99.6366)LR: 1.300e-02
Train: 25 [ 350/390]  Loss: 0.2764 (0.378)  Acc@1: 92.1875 (86.8857)  Acc@5: 98.4375 (99.6172)LR: 1.300e-02
Train: 25 [ 390/390]  Loss: 0.5510 (0.379)  Acc@1: 80.0000 (86.8880)  Acc@5: 97.5000 (99.6200)LR: 1.300e-02
train_acc 86.888000
Valid: 25 [   0/390]  Loss: 0.4382 (0.438)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 25 [  50/390]  Loss: 0.3686 (0.426)  Acc@1: 85.9375 (85.3554)  Acc@5: 100.0000 (99.2647)
Valid: 25 [ 100/390]  Loss: 0.8542 (0.423)  Acc@1: 75.0000 (85.4889)  Acc@5: 98.4375 (99.4121)
Valid: 25 [ 150/390]  Loss: 0.5366 (0.421)  Acc@1: 81.2500 (85.7823)  Acc@5: 100.0000 (99.3688)
Valid: 25 [ 200/390]  Loss: 0.3908 (0.425)  Acc@1: 87.5000 (85.4944)  Acc@5: 98.4375 (99.3859)
Valid: 25 [ 250/390]  Loss: 0.6587 (0.430)  Acc@1: 82.8125 (85.3461)  Acc@5: 96.8750 (99.3464)
Valid: 25 [ 300/390]  Loss: 0.5298 (0.428)  Acc@1: 85.9375 (85.4392)  Acc@5: 98.4375 (99.3096)
Valid: 25 [ 350/390]  Loss: 0.3523 (0.429)  Acc@1: 85.9375 (85.4345)  Acc@5: 100.0000 (99.3234)
Valid: 25 [ 390/390]  Loss: 0.1912 (0.429)  Acc@1: 92.5000 (85.4560)  Acc@5: 100.0000 (99.3120)
valid_acc 85.456000
epoch = 25   
 genotype = Genotype(normal=[('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('skip_connect', 2), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3647, 0.3220, 0.3134],
        [0.4398, 0.2394, 0.3208],
        [0.3554, 0.3141, 0.3305],
        [0.4070, 0.2661, 0.3269],
        [0.4395, 0.2828, 0.2777],
        [0.3762, 0.2795, 0.3443],
        [0.5001, 0.2291, 0.2707],
        [0.4686, 0.2487, 0.2826],
        [0.4960, 0.2299, 0.2741],
        [0.4081, 0.2880, 0.3040],
        [0.4731, 0.2648, 0.2621],
        [0.4692, 0.2756, 0.2552],
        [0.4627, 0.2402, 0.2971],
        [0.5608, 0.1835, 0.2557]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3773, 0.2807, 0.3420],
        [0.2777, 0.3180, 0.4043],
        [0.3682, 0.3075, 0.3242],
        [0.3130, 0.3394, 0.3477],
        [0.3838, 0.3244, 0.2917],
        [0.3631, 0.3285, 0.3084],
        [0.3077, 0.3466, 0.3456],
        [0.3507, 0.3083, 0.3410],
        [0.3752, 0.2802, 0.3446],
        [0.3271, 0.3385, 0.3345],
        [0.3044, 0.3346, 0.3610],
        [0.3396, 0.3161, 0.3443],
        [0.3277, 0.3473, 0.3250],
        [0.3512, 0.3139, 0.3349]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 26 [   0/390]  Loss: 0.8948 (0.895)  Acc@1: 75.0000 (75.0000)  Acc@5: 96.8750 (96.8750)LR: 1.225e-02
Train: 26 [  50/390]  Loss: 0.4820 (0.373)  Acc@1: 82.8125 (86.9485)  Acc@5: 100.0000 (99.5711)LR: 1.225e-02
Train: 26 [ 100/390]  Loss: 0.4166 (0.358)  Acc@1: 87.5000 (87.6083)  Acc@5: 100.0000 (99.5978)LR: 1.225e-02
Train: 26 [ 150/390]  Loss: 0.4756 (0.369)  Acc@1: 82.8125 (87.2310)  Acc@5: 98.4375 (99.6068)LR: 1.225e-02
Train: 26 [ 200/390]  Loss: 0.4577 (0.367)  Acc@1: 84.3750 (87.4611)  Acc@5: 98.4375 (99.6113)LR: 1.225e-02
Train: 26 [ 250/390]  Loss: 0.3111 (0.365)  Acc@1: 87.5000 (87.5747)  Acc@5: 100.0000 (99.6327)LR: 1.225e-02
Train: 26 [ 300/390]  Loss: 0.3221 (0.365)  Acc@1: 87.5000 (87.4066)  Acc@5: 100.0000 (99.6626)LR: 1.225e-02
Train: 26 [ 350/390]  Loss: 0.5859 (0.366)  Acc@1: 84.3750 (87.3887)  Acc@5: 98.4375 (99.6439)LR: 1.225e-02
Train: 26 [ 390/390]  Loss: 0.3790 (0.369)  Acc@1: 85.0000 (87.3160)  Acc@5: 100.0000 (99.6160)LR: 1.225e-02
train_acc 87.316000
Valid: 26 [   0/390]  Loss: 0.5863 (0.586)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 26 [  50/390]  Loss: 0.3379 (0.435)  Acc@1: 87.5000 (85.2328)  Acc@5: 100.0000 (99.1728)
Valid: 26 [ 100/390]  Loss: 0.3506 (0.457)  Acc@1: 85.9375 (84.4988)  Acc@5: 100.0000 (99.0718)
Valid: 26 [ 150/390]  Loss: 0.4020 (0.454)  Acc@1: 87.5000 (84.2198)  Acc@5: 100.0000 (99.0998)
Valid: 26 [ 200/390]  Loss: 0.3969 (0.453)  Acc@1: 87.5000 (84.2662)  Acc@5: 98.4375 (99.1604)
Valid: 26 [ 250/390]  Loss: 0.5531 (0.456)  Acc@1: 79.6875 (84.0326)  Acc@5: 98.4375 (99.1472)
Valid: 26 [ 300/390]  Loss: 0.3001 (0.451)  Acc@1: 92.1875 (84.2452)  Acc@5: 100.0000 (99.2006)
Valid: 26 [ 350/390]  Loss: 0.3691 (0.450)  Acc@1: 89.0625 (84.3661)  Acc@5: 98.4375 (99.2210)
Valid: 26 [ 390/390]  Loss: 0.2350 (0.448)  Acc@1: 95.0000 (84.4480)  Acc@5: 100.0000 (99.2080)
valid_acc 84.448000
epoch = 26   
 genotype = Genotype(normal=[('skip_connect', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3657, 0.3200, 0.3143],
        [0.4456, 0.2359, 0.3185],
        [0.3569, 0.3139, 0.3292],
        [0.4102, 0.2638, 0.3260],
        [0.4441, 0.2811, 0.2748],
        [0.3763, 0.2788, 0.3449],
        [0.5080, 0.2256, 0.2664],
        [0.4715, 0.2462, 0.2823],
        [0.5027, 0.2268, 0.2705],
        [0.4116, 0.2870, 0.3014],
        [0.4797, 0.2621, 0.2583],
        [0.4719, 0.2732, 0.2550],
        [0.4731, 0.2371, 0.2898],
        [0.5685, 0.1798, 0.2518]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3770, 0.2812, 0.3418],
        [0.2796, 0.3187, 0.4017],
        [0.3675, 0.3079, 0.3246],
        [0.3145, 0.3378, 0.3477],
        [0.3852, 0.3234, 0.2914],
        [0.3610, 0.3312, 0.3078],
        [0.3091, 0.3473, 0.3436],
        [0.3499, 0.3044, 0.3457],
        [0.3790, 0.2800, 0.3411],
        [0.3244, 0.3407, 0.3349],
        [0.3039, 0.3356, 0.3605],
        [0.3416, 0.3140, 0.3444],
        [0.3271, 0.3474, 0.3255],
        [0.3505, 0.3137, 0.3358]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 27 [   0/390]  Loss: 0.1562 (0.156)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.150e-02
Train: 27 [  50/390]  Loss: 0.2697 (0.346)  Acc@1: 87.5000 (87.5613)  Acc@5: 100.0000 (99.7549)LR: 1.150e-02
Train: 27 [ 100/390]  Loss: 0.3808 (0.341)  Acc@1: 82.8125 (87.9950)  Acc@5: 98.4375 (99.7061)LR: 1.150e-02
Train: 27 [ 150/390]  Loss: 0.3067 (0.337)  Acc@1: 89.0625 (88.0898)  Acc@5: 100.0000 (99.7206)LR: 1.150e-02
Train: 27 [ 200/390]  Loss: 0.5341 (0.343)  Acc@1: 81.2500 (88.0364)  Acc@5: 96.8750 (99.6891)LR: 1.150e-02
Train: 27 [ 250/390]  Loss: 0.5079 (0.346)  Acc@1: 81.2500 (87.9109)  Acc@5: 100.0000 (99.7074)LR: 1.150e-02
Train: 27 [ 300/390]  Loss: 0.2819 (0.350)  Acc@1: 92.1875 (87.8374)  Acc@5: 98.4375 (99.7041)LR: 1.150e-02
Train: 27 [ 350/390]  Loss: 0.3498 (0.353)  Acc@1: 85.9375 (87.7760)  Acc@5: 100.0000 (99.6795)LR: 1.150e-02
Train: 27 [ 390/390]  Loss: 0.5107 (0.357)  Acc@1: 82.5000 (87.5960)  Acc@5: 100.0000 (99.6600)LR: 1.150e-02
train_acc 87.596000
Valid: 27 [   0/390]  Loss: 0.3651 (0.365)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 27 [  50/390]  Loss: 0.6077 (0.415)  Acc@1: 78.1250 (86.0600)  Acc@5: 96.8750 (99.1728)
Valid: 27 [ 100/390]  Loss: 0.3387 (0.414)  Acc@1: 89.0625 (85.9684)  Acc@5: 100.0000 (99.2265)
Valid: 27 [ 150/390]  Loss: 0.3826 (0.424)  Acc@1: 87.5000 (85.5029)  Acc@5: 100.0000 (99.2136)
Valid: 27 [ 200/390]  Loss: 0.5237 (0.429)  Acc@1: 87.5000 (85.3778)  Acc@5: 96.8750 (99.2226)
Valid: 27 [ 250/390]  Loss: 0.4102 (0.432)  Acc@1: 89.0625 (85.3274)  Acc@5: 100.0000 (99.2094)
Valid: 27 [ 300/390]  Loss: 0.3710 (0.431)  Acc@1: 85.9375 (85.4911)  Acc@5: 100.0000 (99.2473)
Valid: 27 [ 350/390]  Loss: 0.3824 (0.435)  Acc@1: 89.0625 (85.4078)  Acc@5: 98.4375 (99.1898)
Valid: 27 [ 390/390]  Loss: 0.4815 (0.437)  Acc@1: 82.5000 (85.3120)  Acc@5: 100.0000 (99.1800)
valid_acc 85.312000
epoch = 27   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3667, 0.3189, 0.3145],
        [0.4495, 0.2314, 0.3191],
        [0.3571, 0.3123, 0.3306],
        [0.4169, 0.2617, 0.3215],
        [0.4469, 0.2781, 0.2750],
        [0.3806, 0.2798, 0.3397],
        [0.5165, 0.2214, 0.2621],
        [0.4770, 0.2453, 0.2776],
        [0.5115, 0.2225, 0.2661],
        [0.4124, 0.2864, 0.3012],
        [0.4883, 0.2581, 0.2536],
        [0.4793, 0.2711, 0.2496],
        [0.4847, 0.2357, 0.2796],
        [0.5749, 0.1752, 0.2499]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3794, 0.2782, 0.3424],
        [0.2776, 0.3182, 0.4042],
        [0.3674, 0.3058, 0.3268],
        [0.3136, 0.3409, 0.3455],
        [0.3871, 0.3223, 0.2906],
        [0.3644, 0.3290, 0.3066],
        [0.3061, 0.3491, 0.3449],
        [0.3500, 0.3012, 0.3489],
        [0.3820, 0.2780, 0.3400],
        [0.3279, 0.3413, 0.3309],
        [0.3015, 0.3361, 0.3624],
        [0.3422, 0.3121, 0.3457],
        [0.3263, 0.3459, 0.3278],
        [0.3524, 0.3134, 0.3342]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 28 [   0/390]  Loss: 0.3227 (0.323)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.075e-02
Train: 28 [  50/390]  Loss: 0.2978 (0.326)  Acc@1: 92.1875 (88.8480)  Acc@5: 100.0000 (99.6324)LR: 1.075e-02
Train: 28 [ 100/390]  Loss: 0.2701 (0.335)  Acc@1: 92.1875 (88.7222)  Acc@5: 100.0000 (99.7061)LR: 1.075e-02
Train: 28 [ 150/390]  Loss: 0.5321 (0.340)  Acc@1: 81.2500 (88.4623)  Acc@5: 96.8750 (99.6482)LR: 1.075e-02
Train: 28 [ 200/390]  Loss: 0.3779 (0.342)  Acc@1: 87.5000 (88.4251)  Acc@5: 100.0000 (99.6813)LR: 1.075e-02
Train: 28 [ 250/390]  Loss: 0.4975 (0.342)  Acc@1: 85.9375 (88.3093)  Acc@5: 100.0000 (99.6701)LR: 1.075e-02
Train: 28 [ 300/390]  Loss: 0.5709 (0.344)  Acc@1: 79.6875 (88.3409)  Acc@5: 95.3125 (99.6366)LR: 1.075e-02
Train: 28 [ 350/390]  Loss: 0.1954 (0.343)  Acc@1: 93.7500 (88.3280)  Acc@5: 100.0000 (99.6528)LR: 1.075e-02
Train: 28 [ 390/390]  Loss: 0.3520 (0.344)  Acc@1: 85.0000 (88.2480)  Acc@5: 100.0000 (99.6440)LR: 1.075e-02
train_acc 88.248000
Valid: 28 [   0/390]  Loss: 0.5071 (0.507)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 28 [  50/390]  Loss: 0.4720 (0.452)  Acc@1: 82.8125 (83.8542)  Acc@5: 98.4375 (99.2953)
Valid: 28 [ 100/390]  Loss: 0.2209 (0.442)  Acc@1: 92.1875 (84.3131)  Acc@5: 100.0000 (99.3812)
Valid: 28 [ 150/390]  Loss: 0.6583 (0.451)  Acc@1: 76.5625 (84.1784)  Acc@5: 100.0000 (99.3791)
Valid: 28 [ 200/390]  Loss: 0.4703 (0.445)  Acc@1: 84.3750 (84.3983)  Acc@5: 100.0000 (99.4248)
Valid: 28 [ 250/390]  Loss: 0.3523 (0.442)  Acc@1: 87.5000 (84.5306)  Acc@5: 100.0000 (99.4273)
Valid: 28 [ 300/390]  Loss: 0.3590 (0.447)  Acc@1: 87.5000 (84.3646)  Acc@5: 100.0000 (99.4394)
Valid: 28 [ 350/390]  Loss: 0.5849 (0.450)  Acc@1: 75.0000 (84.3127)  Acc@5: 98.4375 (99.4168)
Valid: 28 [ 390/390]  Loss: 0.3474 (0.451)  Acc@1: 90.0000 (84.2960)  Acc@5: 100.0000 (99.4200)
valid_acc 84.296000
epoch = 28   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3681, 0.3154, 0.3165],
        [0.4520, 0.2289, 0.3192],
        [0.3567, 0.3092, 0.3341],
        [0.4256, 0.2584, 0.3160],
        [0.4517, 0.2752, 0.2731],
        [0.3827, 0.2772, 0.3401],
        [0.5213, 0.2172, 0.2615],
        [0.4828, 0.2438, 0.2734],
        [0.5197, 0.2177, 0.2627],
        [0.4157, 0.2844, 0.2999],
        [0.4959, 0.2534, 0.2507],
        [0.4868, 0.2696, 0.2436],
        [0.4942, 0.2331, 0.2727],
        [0.5846, 0.1711, 0.2443]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3803, 0.2763, 0.3435],
        [0.2771, 0.3162, 0.4068],
        [0.3715, 0.3050, 0.3236],
        [0.3095, 0.3417, 0.3488],
        [0.3882, 0.3217, 0.2901],
        [0.3684, 0.3282, 0.3035],
        [0.3054, 0.3512, 0.3433],
        [0.3487, 0.2986, 0.3527],
        [0.3860, 0.2773, 0.3367],
        [0.3252, 0.3435, 0.3312],
        [0.3013, 0.3367, 0.3620],
        [0.3431, 0.3119, 0.3450],
        [0.3252, 0.3469, 0.3279],
        [0.3520, 0.3112, 0.3368]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 29 [   0/390]  Loss: 0.3934 (0.393)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 1.002e-02
Train: 29 [  50/390]  Loss: 0.3523 (0.337)  Acc@1: 84.3750 (88.9400)  Acc@5: 100.0000 (99.6936)LR: 1.002e-02
Train: 29 [ 100/390]  Loss: 0.4000 (0.338)  Acc@1: 85.9375 (88.6912)  Acc@5: 100.0000 (99.7061)LR: 1.002e-02
Train: 29 [ 150/390]  Loss: 0.3833 (0.339)  Acc@1: 85.9375 (88.6072)  Acc@5: 98.4375 (99.7103)LR: 1.002e-02
Train: 29 [ 200/390]  Loss: 0.3230 (0.336)  Acc@1: 92.1875 (88.5961)  Acc@5: 100.0000 (99.6968)LR: 1.002e-02
Train: 29 [ 250/390]  Loss: 0.3540 (0.337)  Acc@1: 87.5000 (88.5022)  Acc@5: 100.0000 (99.7012)LR: 1.002e-02
Train: 29 [ 300/390]  Loss: 0.3275 (0.339)  Acc@1: 89.0625 (88.3461)  Acc@5: 100.0000 (99.6937)LR: 1.002e-02
Train: 29 [ 350/390]  Loss: 0.2581 (0.339)  Acc@1: 92.1875 (88.4660)  Acc@5: 100.0000 (99.7017)LR: 1.002e-02
Train: 29 [ 390/390]  Loss: 0.2998 (0.340)  Acc@1: 90.0000 (88.4280)  Acc@5: 100.0000 (99.6960)LR: 1.002e-02
train_acc 88.428000
Valid: 29 [   0/390]  Loss: 0.5310 (0.531)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 29 [  50/390]  Loss: 0.3381 (0.476)  Acc@1: 89.0625 (83.6397)  Acc@5: 98.4375 (99.0196)
Valid: 29 [ 100/390]  Loss: 0.5021 (0.460)  Acc@1: 82.8125 (84.3750)  Acc@5: 98.4375 (98.9790)
Valid: 29 [ 150/390]  Loss: 0.4932 (0.460)  Acc@1: 81.2500 (84.4785)  Acc@5: 98.4375 (99.0480)
Valid: 29 [ 200/390]  Loss: 0.4403 (0.461)  Acc@1: 85.9375 (84.3983)  Acc@5: 100.0000 (99.0438)
Valid: 29 [ 250/390]  Loss: 0.5339 (0.456)  Acc@1: 76.5625 (84.5244)  Acc@5: 100.0000 (99.0849)
Valid: 29 [ 300/390]  Loss: 0.6129 (0.453)  Acc@1: 75.0000 (84.6709)  Acc@5: 98.4375 (99.0812)
Valid: 29 [ 350/390]  Loss: 0.4031 (0.456)  Acc@1: 85.9375 (84.6465)  Acc@5: 98.4375 (99.0696)
Valid: 29 [ 390/390]  Loss: 0.4306 (0.456)  Acc@1: 82.5000 (84.6920)  Acc@5: 100.0000 (99.0680)
valid_acc 84.692000
epoch = 29   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3698, 0.3142, 0.3159],
        [0.4553, 0.2269, 0.3177],
        [0.3602, 0.3048, 0.3351],
        [0.4335, 0.2568, 0.3097],
        [0.4542, 0.2720, 0.2738],
        [0.3846, 0.2757, 0.3397],
        [0.5260, 0.2130, 0.2610],
        [0.4887, 0.2412, 0.2701],
        [0.5305, 0.2132, 0.2562],
        [0.4205, 0.2826, 0.2969],
        [0.5047, 0.2514, 0.2439],
        [0.4970, 0.2684, 0.2346],
        [0.5058, 0.2290, 0.2652],
        [0.5917, 0.1669, 0.2415]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3806, 0.2742, 0.3452],
        [0.2775, 0.3176, 0.4048],
        [0.3729, 0.3044, 0.3227],
        [0.3074, 0.3421, 0.3505],
        [0.3901, 0.3195, 0.2904],
        [0.3714, 0.3279, 0.3007],
        [0.3048, 0.3500, 0.3452],
        [0.3473, 0.2978, 0.3549],
        [0.3855, 0.2774, 0.3370],
        [0.3246, 0.3431, 0.3323],
        [0.3005, 0.3376, 0.3619],
        [0.3439, 0.3137, 0.3424],
        [0.3244, 0.3496, 0.3260],
        [0.3512, 0.3102, 0.3385]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 30 [   0/390]  Loss: 0.1658 (0.166)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 9.292e-03
Train: 30 [  50/390]  Loss: 0.2874 (0.300)  Acc@1: 87.5000 (90.1654)  Acc@5: 100.0000 (99.7549)LR: 9.292e-03
Train: 30 [ 100/390]  Loss: 0.2247 (0.313)  Acc@1: 95.3125 (89.3719)  Acc@5: 100.0000 (99.6597)LR: 9.292e-03
Train: 30 [ 150/390]  Loss: 0.3274 (0.324)  Acc@1: 89.0625 (88.9176)  Acc@5: 100.0000 (99.5861)LR: 9.292e-03
Train: 30 [ 200/390]  Loss: 0.1587 (0.322)  Acc@1: 96.8750 (88.9692)  Acc@5: 100.0000 (99.6035)LR: 9.292e-03
Train: 30 [ 250/390]  Loss: 0.5272 (0.327)  Acc@1: 85.9375 (88.7948)  Acc@5: 100.0000 (99.6140)LR: 9.292e-03
Train: 30 [ 300/390]  Loss: 0.4671 (0.329)  Acc@1: 85.9375 (88.6836)  Acc@5: 98.4375 (99.6055)LR: 9.292e-03
Train: 30 [ 350/390]  Loss: 0.2701 (0.329)  Acc@1: 85.9375 (88.6841)  Acc@5: 100.0000 (99.6216)LR: 9.292e-03
Train: 30 [ 390/390]  Loss: 0.3054 (0.330)  Acc@1: 90.0000 (88.6040)  Acc@5: 100.0000 (99.6320)LR: 9.292e-03
train_acc 88.604000
Valid: 30 [   0/390]  Loss: 0.6402 (0.640)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 30 [  50/390]  Loss: 0.4886 (0.511)  Acc@1: 81.2500 (82.0772)  Acc@5: 96.8750 (98.9583)
Valid: 30 [ 100/390]  Loss: 0.3803 (0.522)  Acc@1: 89.0625 (81.6522)  Acc@5: 100.0000 (99.0563)
Valid: 30 [ 150/390]  Loss: 0.3254 (0.515)  Acc@1: 89.0625 (82.0882)  Acc@5: 100.0000 (99.0894)
Valid: 30 [ 200/390]  Loss: 0.4523 (0.512)  Acc@1: 81.2500 (82.2528)  Acc@5: 100.0000 (99.0438)
Valid: 30 [ 250/390]  Loss: 0.5895 (0.509)  Acc@1: 76.5625 (82.3892)  Acc@5: 98.4375 (98.9978)
Valid: 30 [ 300/390]  Loss: 0.6570 (0.508)  Acc@1: 76.5625 (82.3609)  Acc@5: 98.4375 (98.9981)
Valid: 30 [ 350/390]  Loss: 0.6687 (0.512)  Acc@1: 78.1250 (82.3584)  Acc@5: 96.8750 (98.9539)
Valid: 30 [ 390/390]  Loss: 0.4926 (0.512)  Acc@1: 85.0000 (82.3760)  Acc@5: 97.5000 (98.9080)
valid_acc 82.376000
epoch = 30   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3718, 0.3086, 0.3196],
        [0.4574, 0.2257, 0.3169],
        [0.3606, 0.2999, 0.3395],
        [0.4404, 0.2535, 0.3061],
        [0.4592, 0.2691, 0.2717],
        [0.3875, 0.2740, 0.3384],
        [0.5295, 0.2106, 0.2598],
        [0.4923, 0.2396, 0.2681],
        [0.5352, 0.2103, 0.2545],
        [0.4274, 0.2825, 0.2901],
        [0.5114, 0.2464, 0.2422],
        [0.5053, 0.2670, 0.2277],
        [0.5167, 0.2268, 0.2565],
        [0.5987, 0.1632, 0.2381]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3848, 0.2737, 0.3414],
        [0.2775, 0.3173, 0.4052],
        [0.3700, 0.3055, 0.3245],
        [0.3088, 0.3406, 0.3506],
        [0.3903, 0.3186, 0.2910],
        [0.3707, 0.3286, 0.3007],
        [0.3034, 0.3502, 0.3463],
        [0.3480, 0.2958, 0.3563],
        [0.3877, 0.2764, 0.3359],
        [0.3266, 0.3409, 0.3326],
        [0.3005, 0.3380, 0.3615],
        [0.3470, 0.3117, 0.3413],
        [0.3214, 0.3479, 0.3307],
        [0.3490, 0.3079, 0.3431]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 31 [   0/390]  Loss: 0.3212 (0.321)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 8.583e-03
Train: 31 [  50/390]  Loss: 0.2820 (0.345)  Acc@1: 90.6250 (87.5919)  Acc@5: 100.0000 (99.6630)LR: 8.583e-03
Train: 31 [ 100/390]  Loss: 0.5185 (0.357)  Acc@1: 85.9375 (87.4536)  Acc@5: 98.4375 (99.6442)LR: 8.583e-03
Train: 31 [ 150/390]  Loss: 0.3448 (0.341)  Acc@1: 87.5000 (88.2243)  Acc@5: 100.0000 (99.6482)LR: 8.583e-03
Train: 31 [ 200/390]  Loss: 0.3591 (0.333)  Acc@1: 85.9375 (88.5650)  Acc@5: 100.0000 (99.6424)LR: 8.583e-03
Train: 31 [ 250/390]  Loss: 0.2762 (0.329)  Acc@1: 89.0625 (88.6765)  Acc@5: 100.0000 (99.6514)LR: 8.583e-03
Train: 31 [ 300/390]  Loss: 0.2340 (0.331)  Acc@1: 90.6250 (88.4967)  Acc@5: 100.0000 (99.6574)LR: 8.583e-03
Train: 31 [ 350/390]  Loss: 0.3888 (0.332)  Acc@1: 90.6250 (88.5150)  Acc@5: 96.8750 (99.6439)LR: 8.583e-03
Train: 31 [ 390/390]  Loss: 0.4173 (0.334)  Acc@1: 90.0000 (88.3520)  Acc@5: 100.0000 (99.6520)LR: 8.583e-03
train_acc 88.352000
Valid: 31 [   0/390]  Loss: 0.2963 (0.296)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)
Valid: 31 [  50/390]  Loss: 0.3961 (0.428)  Acc@1: 87.5000 (85.7843)  Acc@5: 96.8750 (99.2341)
Valid: 31 [ 100/390]  Loss: 0.4803 (0.434)  Acc@1: 82.8125 (85.4734)  Acc@5: 100.0000 (99.2884)
Valid: 31 [ 150/390]  Loss: 0.4145 (0.437)  Acc@1: 90.6250 (85.4201)  Acc@5: 96.8750 (99.2343)
Valid: 31 [ 200/390]  Loss: 0.2924 (0.437)  Acc@1: 92.1875 (85.3001)  Acc@5: 100.0000 (99.2071)
Valid: 31 [ 250/390]  Loss: 0.5325 (0.443)  Acc@1: 81.2500 (85.0847)  Acc@5: 98.4375 (99.1907)
Valid: 31 [ 300/390]  Loss: 0.5217 (0.445)  Acc@1: 81.2500 (85.0291)  Acc@5: 100.0000 (99.1798)
Valid: 31 [ 350/390]  Loss: 0.4040 (0.445)  Acc@1: 87.5000 (85.0828)  Acc@5: 100.0000 (99.1854)
Valid: 31 [ 390/390]  Loss: 0.6389 (0.444)  Acc@1: 80.0000 (85.1200)  Acc@5: 97.5000 (99.1600)
valid_acc 85.120000
epoch = 31   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3731, 0.3063, 0.3206],
        [0.4620, 0.2238, 0.3142],
        [0.3613, 0.2994, 0.3394],
        [0.4461, 0.2509, 0.3030],
        [0.4615, 0.2654, 0.2731],
        [0.3884, 0.2754, 0.3362],
        [0.5354, 0.2061, 0.2584],
        [0.4945, 0.2360, 0.2695],
        [0.5422, 0.2076, 0.2502],
        [0.4301, 0.2816, 0.2882],
        [0.5191, 0.2435, 0.2375],
        [0.5114, 0.2651, 0.2235],
        [0.5245, 0.2247, 0.2508],
        [0.6028, 0.1604, 0.2369]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3866, 0.2727, 0.3407],
        [0.2771, 0.3171, 0.4058],
        [0.3708, 0.3062, 0.3230],
        [0.3081, 0.3395, 0.3524],
        [0.3908, 0.3159, 0.2933],
        [0.3690, 0.3268, 0.3042],
        [0.3049, 0.3501, 0.3450],
        [0.3482, 0.2956, 0.3562],
        [0.3893, 0.2783, 0.3324],
        [0.3252, 0.3429, 0.3318],
        [0.3000, 0.3395, 0.3605],
        [0.3479, 0.3114, 0.3407],
        [0.3219, 0.3481, 0.3300],
        [0.3489, 0.3064, 0.3447]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 32 [   0/390]  Loss: 0.2873 (0.287)  Acc@1: 90.6250 (90.6250)  Acc@5: 98.4375 (98.4375)LR: 7.891e-03
Train: 32 [  50/390]  Loss: 0.2115 (0.317)  Acc@1: 93.7500 (88.8480)  Acc@5: 100.0000 (99.5711)LR: 7.891e-03
Train: 32 [ 100/390]  Loss: 0.5199 (0.296)  Acc@1: 85.9375 (89.4647)  Acc@5: 100.0000 (99.6906)LR: 7.891e-03
Train: 32 [ 150/390]  Loss: 0.4146 (0.299)  Acc@1: 85.9375 (89.4557)  Acc@5: 100.0000 (99.6896)LR: 7.891e-03
Train: 32 [ 200/390]  Loss: 0.2441 (0.306)  Acc@1: 92.1875 (89.4123)  Acc@5: 100.0000 (99.6735)LR: 7.891e-03
Train: 32 [ 250/390]  Loss: 0.4333 (0.308)  Acc@1: 84.3750 (89.3115)  Acc@5: 100.0000 (99.7136)LR: 7.891e-03
Train: 32 [ 300/390]  Loss: 0.3652 (0.310)  Acc@1: 85.9375 (89.2805)  Acc@5: 100.0000 (99.7353)LR: 7.891e-03
Train: 32 [ 350/390]  Loss: 0.2975 (0.307)  Acc@1: 90.6250 (89.3385)  Acc@5: 100.0000 (99.7196)LR: 7.891e-03
Train: 32 [ 390/390]  Loss: 0.3426 (0.310)  Acc@1: 85.0000 (89.2440)  Acc@5: 100.0000 (99.7120)LR: 7.891e-03
train_acc 89.244000
Valid: 32 [   0/390]  Loss: 0.2935 (0.294)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 32 [  50/390]  Loss: 0.4459 (0.471)  Acc@1: 85.9375 (84.2525)  Acc@5: 100.0000 (99.1728)
Valid: 32 [ 100/390]  Loss: 0.4532 (0.466)  Acc@1: 81.2500 (84.0501)  Acc@5: 98.4375 (99.2884)
Valid: 32 [ 150/390]  Loss: 0.3618 (0.459)  Acc@1: 87.5000 (84.4060)  Acc@5: 100.0000 (99.3584)
Valid: 32 [ 200/390]  Loss: 0.3482 (0.460)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (99.3237)
Valid: 32 [ 250/390]  Loss: 0.6410 (0.461)  Acc@1: 79.6875 (84.3252)  Acc@5: 100.0000 (99.2779)
Valid: 32 [ 300/390]  Loss: 0.5037 (0.462)  Acc@1: 82.8125 (84.2816)  Acc@5: 96.8750 (99.2733)
Valid: 32 [ 350/390]  Loss: 0.3531 (0.463)  Acc@1: 87.5000 (84.2504)  Acc@5: 100.0000 (99.2655)
Valid: 32 [ 390/390]  Loss: 0.5686 (0.462)  Acc@1: 77.5000 (84.2120)  Acc@5: 100.0000 (99.2680)
valid_acc 84.212000
epoch = 32   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3759, 0.3024, 0.3218],
        [0.4687, 0.2203, 0.3110],
        [0.3630, 0.2983, 0.3387],
        [0.4539, 0.2471, 0.2990],
        [0.4689, 0.2625, 0.2686],
        [0.3920, 0.2739, 0.3341],
        [0.5422, 0.2020, 0.2558],
        [0.5012, 0.2327, 0.2661],
        [0.5525, 0.2030, 0.2445],
        [0.4351, 0.2804, 0.2845],
        [0.5294, 0.2395, 0.2311],
        [0.5212, 0.2637, 0.2152],
        [0.5354, 0.2195, 0.2451],
        [0.6134, 0.1560, 0.2307]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3882, 0.2707, 0.3411],
        [0.2782, 0.3164, 0.4054],
        [0.3700, 0.3068, 0.3231],
        [0.3088, 0.3384, 0.3528],
        [0.3921, 0.3134, 0.2945],
        [0.3683, 0.3268, 0.3050],
        [0.3054, 0.3485, 0.3461],
        [0.3509, 0.2943, 0.3548],
        [0.3941, 0.2774, 0.3286],
        [0.3234, 0.3438, 0.3328],
        [0.2991, 0.3428, 0.3581],
        [0.3481, 0.3079, 0.3441],
        [0.3177, 0.3511, 0.3312],
        [0.3492, 0.3029, 0.3479]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 33 [   0/390]  Loss: 0.2883 (0.288)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 7.219e-03
Train: 33 [  50/390]  Loss: 0.3456 (0.301)  Acc@1: 84.3750 (89.7365)  Acc@5: 100.0000 (99.8162)LR: 7.219e-03
Train: 33 [ 100/390]  Loss: 0.4038 (0.305)  Acc@1: 89.0625 (89.6968)  Acc@5: 98.4375 (99.7834)LR: 7.219e-03
Train: 33 [ 150/390]  Loss: 0.3449 (0.309)  Acc@1: 85.9375 (89.5281)  Acc@5: 100.0000 (99.7310)LR: 7.219e-03
Train: 33 [ 200/390]  Loss: 0.3164 (0.306)  Acc@1: 89.0625 (89.5833)  Acc@5: 100.0000 (99.7590)LR: 7.219e-03
Train: 33 [ 250/390]  Loss: 0.3057 (0.308)  Acc@1: 85.9375 (89.5792)  Acc@5: 98.4375 (99.7136)LR: 7.219e-03
Train: 33 [ 300/390]  Loss: 0.4618 (0.312)  Acc@1: 82.8125 (89.3480)  Acc@5: 100.0000 (99.7197)LR: 7.219e-03
Train: 33 [ 350/390]  Loss: 0.2210 (0.315)  Acc@1: 92.1875 (89.2406)  Acc@5: 100.0000 (99.7062)LR: 7.219e-03
Train: 33 [ 390/390]  Loss: 0.2414 (0.315)  Acc@1: 92.5000 (89.2040)  Acc@5: 97.5000 (99.7120)LR: 7.219e-03
train_acc 89.204000
Valid: 33 [   0/390]  Loss: 0.5511 (0.551)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 33 [  50/390]  Loss: 0.4246 (0.460)  Acc@1: 85.9375 (84.5895)  Acc@5: 100.0000 (99.2034)
Valid: 33 [ 100/390]  Loss: 0.6622 (0.472)  Acc@1: 78.1250 (84.0965)  Acc@5: 98.4375 (99.1027)
Valid: 33 [ 150/390]  Loss: 0.5354 (0.479)  Acc@1: 81.2500 (83.5782)  Acc@5: 100.0000 (99.0791)
Valid: 33 [ 200/390]  Loss: 0.6349 (0.475)  Acc@1: 82.8125 (83.8464)  Acc@5: 98.4375 (99.1138)
Valid: 33 [ 250/390]  Loss: 0.6393 (0.473)  Acc@1: 82.8125 (83.9206)  Acc@5: 100.0000 (99.1658)
Valid: 33 [ 300/390]  Loss: 0.5592 (0.472)  Acc@1: 73.4375 (83.9234)  Acc@5: 96.8750 (99.1694)
Valid: 33 [ 350/390]  Loss: 0.1904 (0.468)  Acc@1: 95.3125 (83.9966)  Acc@5: 100.0000 (99.2032)
Valid: 33 [ 390/390]  Loss: 0.4187 (0.469)  Acc@1: 87.5000 (83.9640)  Acc@5: 100.0000 (99.2040)
valid_acc 83.964000
epoch = 33   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3773, 0.3007, 0.3220],
        [0.4719, 0.2187, 0.3094],
        [0.3653, 0.2987, 0.3360],
        [0.4608, 0.2451, 0.2941],
        [0.4726, 0.2600, 0.2674],
        [0.3930, 0.2743, 0.3327],
        [0.5488, 0.2004, 0.2508],
        [0.5056, 0.2319, 0.2625],
        [0.5585, 0.2008, 0.2407],
        [0.4369, 0.2801, 0.2830],
        [0.5372, 0.2376, 0.2252],
        [0.5269, 0.2610, 0.2120],
        [0.5440, 0.2153, 0.2406],
        [0.6235, 0.1520, 0.2245]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3913, 0.2708, 0.3380],
        [0.2768, 0.3197, 0.4035],
        [0.3698, 0.3068, 0.3234],
        [0.3080, 0.3378, 0.3542],
        [0.3912, 0.3107, 0.2981],
        [0.3670, 0.3256, 0.3074],
        [0.3049, 0.3480, 0.3471],
        [0.3498, 0.2937, 0.3565],
        [0.3954, 0.2787, 0.3259],
        [0.3216, 0.3459, 0.3325],
        [0.3008, 0.3415, 0.3576],
        [0.3488, 0.3079, 0.3434],
        [0.3199, 0.3529, 0.3272],
        [0.3513, 0.3019, 0.3468]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 34 [   0/390]  Loss: 0.2756 (0.276)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 6.570e-03
Train: 34 [  50/390]  Loss: 0.2078 (0.316)  Acc@1: 90.6250 (89.4914)  Acc@5: 100.0000 (99.6936)LR: 6.570e-03
Train: 34 [ 100/390]  Loss: 0.3843 (0.301)  Acc@1: 87.5000 (89.6968)  Acc@5: 100.0000 (99.6751)LR: 6.570e-03
Train: 34 [ 150/390]  Loss: 0.2576 (0.300)  Acc@1: 92.1875 (89.6523)  Acc@5: 100.0000 (99.6896)LR: 6.570e-03
Train: 34 [ 200/390]  Loss: 0.1877 (0.300)  Acc@1: 95.3125 (89.5833)  Acc@5: 100.0000 (99.7201)LR: 6.570e-03
Train: 34 [ 250/390]  Loss: 0.3300 (0.304)  Acc@1: 89.0625 (89.5667)  Acc@5: 98.4375 (99.7261)LR: 6.570e-03
Train: 34 [ 300/390]  Loss: 0.3739 (0.303)  Acc@1: 89.0625 (89.6647)  Acc@5: 100.0000 (99.7404)LR: 6.570e-03
Train: 34 [ 350/390]  Loss: 0.2138 (0.301)  Acc@1: 93.7500 (89.7569)  Acc@5: 100.0000 (99.7463)LR: 6.570e-03
Train: 34 [ 390/390]  Loss: 0.7009 (0.305)  Acc@1: 75.0000 (89.5720)  Acc@5: 100.0000 (99.7400)LR: 6.570e-03
train_acc 89.572000
Valid: 34 [   0/390]  Loss: 0.4733 (0.473)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 34 [  50/390]  Loss: 0.4012 (0.485)  Acc@1: 87.5000 (84.0993)  Acc@5: 100.0000 (98.9890)
Valid: 34 [ 100/390]  Loss: 0.5498 (0.488)  Acc@1: 82.8125 (83.9728)  Acc@5: 98.4375 (99.1337)
Valid: 34 [ 150/390]  Loss: 0.4989 (0.486)  Acc@1: 87.5000 (84.0025)  Acc@5: 100.0000 (99.0377)
Valid: 34 [ 200/390]  Loss: 0.2379 (0.485)  Acc@1: 92.1875 (83.7764)  Acc@5: 100.0000 (99.0516)
Valid: 34 [ 250/390]  Loss: 0.3375 (0.482)  Acc@1: 92.1875 (83.9206)  Acc@5: 100.0000 (99.1223)
Valid: 34 [ 300/390]  Loss: 0.4707 (0.482)  Acc@1: 81.2500 (83.8870)  Acc@5: 100.0000 (99.1383)
Valid: 34 [ 350/390]  Loss: 0.5585 (0.483)  Acc@1: 82.8125 (83.7384)  Acc@5: 100.0000 (99.1720)
Valid: 34 [ 390/390]  Loss: 0.3171 (0.482)  Acc@1: 87.5000 (83.7480)  Acc@5: 100.0000 (99.1920)
valid_acc 83.748000
epoch = 34   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('skip_connect', 3), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3808, 0.2978, 0.3214],
        [0.4760, 0.2169, 0.3071],
        [0.3697, 0.2974, 0.3328],
        [0.4672, 0.2415, 0.2912],
        [0.4784, 0.2570, 0.2646],
        [0.3960, 0.2712, 0.3327],
        [0.5553, 0.1980, 0.2467],
        [0.5126, 0.2282, 0.2592],
        [0.5685, 0.1962, 0.2353],
        [0.4411, 0.2771, 0.2817],
        [0.5432, 0.2341, 0.2227],
        [0.5379, 0.2561, 0.2060],
        [0.5534, 0.2110, 0.2355],
        [0.6315, 0.1497, 0.2189]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3922, 0.2729, 0.3349],
        [0.2794, 0.3200, 0.4006],
        [0.3700, 0.3106, 0.3194],
        [0.3068, 0.3414, 0.3518],
        [0.3906, 0.3061, 0.3033],
        [0.3670, 0.3275, 0.3055],
        [0.3039, 0.3505, 0.3456],
        [0.3529, 0.2918, 0.3553],
        [0.3952, 0.2809, 0.3240],
        [0.3203, 0.3449, 0.3348],
        [0.3004, 0.3446, 0.3550],
        [0.3510, 0.3056, 0.3433],
        [0.3182, 0.3558, 0.3260],
        [0.3495, 0.3005, 0.3500]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 35 [   0/390]  Loss: 0.1670 (0.167)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 5.947e-03
Train: 35 [  50/390]  Loss: 0.1947 (0.293)  Acc@1: 93.7500 (90.0123)  Acc@5: 98.4375 (99.6936)LR: 5.947e-03
Train: 35 [ 100/390]  Loss: 0.2064 (0.299)  Acc@1: 92.1875 (89.2946)  Acc@5: 100.0000 (99.7370)LR: 5.947e-03
Train: 35 [ 150/390]  Loss: 0.4379 (0.300)  Acc@1: 85.9375 (89.3419)  Acc@5: 98.4375 (99.7620)LR: 5.947e-03
Train: 35 [ 200/390]  Loss: 0.1873 (0.299)  Acc@1: 92.1875 (89.5522)  Acc@5: 100.0000 (99.7668)LR: 5.947e-03
Train: 35 [ 250/390]  Loss: 0.1958 (0.298)  Acc@1: 95.3125 (89.6228)  Acc@5: 100.0000 (99.7697)LR: 5.947e-03
Train: 35 [ 300/390]  Loss: 0.2119 (0.296)  Acc@1: 92.1875 (89.7737)  Acc@5: 100.0000 (99.7716)LR: 5.947e-03
Train: 35 [ 350/390]  Loss: 0.2154 (0.294)  Acc@1: 92.1875 (89.8282)  Acc@5: 100.0000 (99.7418)LR: 5.947e-03
Train: 35 [ 390/390]  Loss: 0.2617 (0.295)  Acc@1: 92.5000 (89.8000)  Acc@5: 100.0000 (99.7520)LR: 5.947e-03
train_acc 89.800000
Valid: 35 [   0/390]  Loss: 0.4015 (0.401)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)
Valid: 35 [  50/390]  Loss: 0.3896 (0.498)  Acc@1: 92.1875 (82.6900)  Acc@5: 98.4375 (99.2953)
Valid: 35 [ 100/390]  Loss: 0.6229 (0.483)  Acc@1: 75.0000 (83.3230)  Acc@5: 98.4375 (99.0718)
Valid: 35 [ 150/390]  Loss: 0.5244 (0.476)  Acc@1: 85.9375 (83.6403)  Acc@5: 98.4375 (99.2239)
Valid: 35 [ 200/390]  Loss: 0.5329 (0.472)  Acc@1: 82.8125 (83.7531)  Acc@5: 100.0000 (99.2460)
Valid: 35 [ 250/390]  Loss: 0.4673 (0.467)  Acc@1: 82.8125 (83.9828)  Acc@5: 100.0000 (99.2717)
Valid: 35 [ 300/390]  Loss: 0.3194 (0.468)  Acc@1: 89.0625 (83.9078)  Acc@5: 100.0000 (99.2369)
Valid: 35 [ 350/390]  Loss: 0.5879 (0.464)  Acc@1: 75.0000 (84.0322)  Acc@5: 100.0000 (99.2477)
Valid: 35 [ 390/390]  Loss: 0.4821 (0.465)  Acc@1: 90.0000 (83.9640)  Acc@5: 100.0000 (99.2560)
valid_acc 83.964000
epoch = 35   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3842, 0.2956, 0.3203],
        [0.4800, 0.2140, 0.3060],
        [0.3719, 0.2955, 0.3326],
        [0.4750, 0.2383, 0.2867],
        [0.4817, 0.2530, 0.2653],
        [0.4018, 0.2694, 0.3288],
        [0.5615, 0.1955, 0.2430],
        [0.5162, 0.2248, 0.2590],
        [0.5752, 0.1922, 0.2326],
        [0.4433, 0.2747, 0.2819],
        [0.5481, 0.2318, 0.2201],
        [0.5430, 0.2543, 0.2026],
        [0.5600, 0.2071, 0.2329],
        [0.6387, 0.1461, 0.2152]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3913, 0.2723, 0.3364],
        [0.2810, 0.3200, 0.3990],
        [0.3742, 0.3078, 0.3180],
        [0.3062, 0.3407, 0.3531],
        [0.3903, 0.3038, 0.3059],
        [0.3659, 0.3258, 0.3083],
        [0.3056, 0.3506, 0.3438],
        [0.3540, 0.2916, 0.3544],
        [0.3976, 0.2800, 0.3223],
        [0.3201, 0.3434, 0.3365],
        [0.2967, 0.3444, 0.3589],
        [0.3524, 0.3053, 0.3423],
        [0.3168, 0.3530, 0.3301],
        [0.3495, 0.2993, 0.3512]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 36 [   0/390]  Loss: 0.4691 (0.469)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)LR: 5.351e-03
Train: 36 [  50/390]  Loss: 0.3334 (0.277)  Acc@1: 89.0625 (90.1961)  Acc@5: 100.0000 (99.8468)LR: 5.351e-03
Train: 36 [ 100/390]  Loss: 0.2432 (0.284)  Acc@1: 92.1875 (90.0681)  Acc@5: 100.0000 (99.8298)LR: 5.351e-03
Train: 36 [ 150/390]  Loss: 0.2998 (0.289)  Acc@1: 85.9375 (89.8903)  Acc@5: 100.0000 (99.7620)LR: 5.351e-03
Train: 36 [ 200/390]  Loss: 0.3015 (0.291)  Acc@1: 89.0625 (89.7777)  Acc@5: 100.0000 (99.7590)LR: 5.351e-03
Train: 36 [ 250/390]  Loss: 0.5099 (0.295)  Acc@1: 89.0625 (89.6352)  Acc@5: 98.4375 (99.7634)LR: 5.351e-03
Train: 36 [ 300/390]  Loss: 0.4201 (0.294)  Acc@1: 89.0625 (89.7529)  Acc@5: 98.4375 (99.7353)LR: 5.351e-03
Train: 36 [ 350/390]  Loss: 0.4734 (0.297)  Acc@1: 85.9375 (89.6189)  Acc@5: 100.0000 (99.7463)LR: 5.351e-03
Train: 36 [ 390/390]  Loss: 0.4253 (0.297)  Acc@1: 85.0000 (89.6200)  Acc@5: 100.0000 (99.7520)LR: 5.351e-03
train_acc 89.620000
Valid: 36 [   0/390]  Loss: 0.6356 (0.636)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)
Valid: 36 [  50/390]  Loss: 0.5221 (0.554)  Acc@1: 82.8125 (80.7598)  Acc@5: 100.0000 (98.7745)
Valid: 36 [ 100/390]  Loss: 0.5395 (0.547)  Acc@1: 82.8125 (80.9715)  Acc@5: 100.0000 (98.9171)
Valid: 36 [ 150/390]  Loss: 0.5217 (0.551)  Acc@1: 81.2500 (80.9706)  Acc@5: 98.4375 (98.8411)
Valid: 36 [ 200/390]  Loss: 0.6254 (0.551)  Acc@1: 75.0000 (81.0090)  Acc@5: 100.0000 (98.8262)
Valid: 36 [ 250/390]  Loss: 0.7565 (0.557)  Acc@1: 71.8750 (80.8267)  Acc@5: 98.4375 (98.7737)
Valid: 36 [ 300/390]  Loss: 0.6398 (0.560)  Acc@1: 81.2500 (80.7205)  Acc@5: 98.4375 (98.7386)
Valid: 36 [ 350/390]  Loss: 0.4461 (0.562)  Acc@1: 81.2500 (80.5867)  Acc@5: 98.4375 (98.7758)
Valid: 36 [ 390/390]  Loss: 0.6872 (0.563)  Acc@1: 77.5000 (80.5600)  Acc@5: 97.5000 (98.7320)
valid_acc 80.560000
epoch = 36   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3872, 0.2924, 0.3204],
        [0.4837, 0.2096, 0.3068],
        [0.3746, 0.2943, 0.3311],
        [0.4827, 0.2329, 0.2844],
        [0.4860, 0.2509, 0.2631],
        [0.4061, 0.2677, 0.3262],
        [0.5662, 0.1918, 0.2420],
        [0.5218, 0.2228, 0.2554],
        [0.5804, 0.1891, 0.2305],
        [0.4479, 0.2732, 0.2789],
        [0.5533, 0.2284, 0.2183],
        [0.5500, 0.2536, 0.1965],
        [0.5632, 0.2039, 0.2329],
        [0.6431, 0.1441, 0.2128]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3953, 0.2703, 0.3344],
        [0.2819, 0.3235, 0.3946],
        [0.3719, 0.3090, 0.3191],
        [0.3048, 0.3433, 0.3519],
        [0.3931, 0.2995, 0.3074],
        [0.3653, 0.3272, 0.3075],
        [0.3057, 0.3543, 0.3399],
        [0.3562, 0.2891, 0.3548],
        [0.3981, 0.2804, 0.3215],
        [0.3196, 0.3442, 0.3362],
        [0.2957, 0.3427, 0.3616],
        [0.3545, 0.3039, 0.3416],
        [0.3145, 0.3561, 0.3293],
        [0.3481, 0.2972, 0.3547]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 37 [   0/390]  Loss: 0.6112 (0.611)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)LR: 4.785e-03
Train: 37 [  50/390]  Loss: 0.2222 (0.310)  Acc@1: 92.1875 (88.5417)  Acc@5: 100.0000 (99.8468)LR: 4.785e-03
Train: 37 [ 100/390]  Loss: 0.2750 (0.299)  Acc@1: 92.1875 (89.0780)  Acc@5: 100.0000 (99.8608)LR: 4.785e-03
Train: 37 [ 150/390]  Loss: 0.2786 (0.292)  Acc@1: 92.1875 (89.6627)  Acc@5: 100.0000 (99.8551)LR: 4.785e-03
Train: 37 [ 200/390]  Loss: 0.3380 (0.284)  Acc@1: 87.5000 (90.0575)  Acc@5: 100.0000 (99.8601)LR: 4.785e-03
Train: 37 [ 250/390]  Loss: 0.2730 (0.290)  Acc@1: 93.7500 (89.9838)  Acc@5: 100.0000 (99.8195)LR: 4.785e-03
Train: 37 [ 300/390]  Loss: 0.2505 (0.293)  Acc@1: 90.6250 (89.9346)  Acc@5: 100.0000 (99.8027)LR: 4.785e-03
Train: 37 [ 350/390]  Loss: 0.4653 (0.292)  Acc@1: 90.6250 (89.9751)  Acc@5: 100.0000 (99.8175)LR: 4.785e-03
Train: 37 [ 390/390]  Loss: 0.3403 (0.293)  Acc@1: 85.0000 (89.8440)  Acc@5: 100.0000 (99.8160)LR: 4.785e-03
train_acc 89.844000
Valid: 37 [   0/390]  Loss: 0.4720 (0.472)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 37 [  50/390]  Loss: 0.4239 (0.481)  Acc@1: 85.9375 (83.7010)  Acc@5: 98.4375 (99.1115)
Valid: 37 [ 100/390]  Loss: 0.5844 (0.506)  Acc@1: 82.8125 (82.5650)  Acc@5: 96.8750 (98.8861)
Valid: 37 [ 150/390]  Loss: 0.4701 (0.504)  Acc@1: 85.9375 (82.5642)  Acc@5: 100.0000 (98.8100)
Valid: 37 [ 200/390]  Loss: 0.5471 (0.510)  Acc@1: 84.3750 (82.4782)  Acc@5: 98.4375 (98.7795)
Valid: 37 [ 250/390]  Loss: 0.5791 (0.513)  Acc@1: 82.8125 (82.4390)  Acc@5: 98.4375 (98.7301)
Valid: 37 [ 300/390]  Loss: 0.5436 (0.513)  Acc@1: 81.2500 (82.4647)  Acc@5: 100.0000 (98.7749)
Valid: 37 [ 350/390]  Loss: 0.5580 (0.509)  Acc@1: 84.3750 (82.6834)  Acc@5: 100.0000 (98.7981)
Valid: 37 [ 390/390]  Loss: 0.3629 (0.507)  Acc@1: 90.0000 (82.8000)  Acc@5: 97.5000 (98.8120)
valid_acc 82.800000
epoch = 37   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3927, 0.2882, 0.3191],
        [0.4842, 0.2084, 0.3074],
        [0.3776, 0.2959, 0.3265],
        [0.4884, 0.2313, 0.2802],
        [0.4891, 0.2490, 0.2619],
        [0.4074, 0.2640, 0.3286],
        [0.5715, 0.1909, 0.2377],
        [0.5282, 0.2204, 0.2514],
        [0.5868, 0.1866, 0.2266],
        [0.4507, 0.2712, 0.2780],
        [0.5586, 0.2257, 0.2157],
        [0.5562, 0.2510, 0.1928],
        [0.5732, 0.2005, 0.2264],
        [0.6469, 0.1417, 0.2114]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3978, 0.2692, 0.3330],
        [0.2807, 0.3242, 0.3952],
        [0.3712, 0.3106, 0.3182],
        [0.3036, 0.3440, 0.3524],
        [0.3947, 0.2971, 0.3082],
        [0.3631, 0.3282, 0.3087],
        [0.3049, 0.3601, 0.3350],
        [0.3557, 0.2875, 0.3568],
        [0.3989, 0.2820, 0.3191],
        [0.3208, 0.3444, 0.3348],
        [0.2970, 0.3427, 0.3603],
        [0.3551, 0.3037, 0.3412],
        [0.3151, 0.3580, 0.3269],
        [0.3472, 0.2980, 0.3548]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 38 [   0/390]  Loss: 0.3279 (0.328)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 4.252e-03
Train: 38 [  50/390]  Loss: 0.3319 (0.284)  Acc@1: 85.9375 (90.1348)  Acc@5: 100.0000 (99.8775)LR: 4.252e-03
Train: 38 [ 100/390]  Loss: 0.1930 (0.295)  Acc@1: 95.3125 (90.0062)  Acc@5: 100.0000 (99.8144)LR: 4.252e-03
Train: 38 [ 150/390]  Loss: 0.1524 (0.298)  Acc@1: 96.8750 (89.8903)  Acc@5: 100.0000 (99.7724)LR: 4.252e-03
Train: 38 [ 200/390]  Loss: 0.2794 (0.295)  Acc@1: 92.1875 (90.0498)  Acc@5: 100.0000 (99.7901)LR: 4.252e-03
Train: 38 [ 250/390]  Loss: 0.1921 (0.294)  Acc@1: 95.3125 (90.0212)  Acc@5: 100.0000 (99.7759)LR: 4.252e-03
Train: 38 [ 300/390]  Loss: 0.1567 (0.292)  Acc@1: 93.7500 (90.0799)  Acc@5: 100.0000 (99.7820)LR: 4.252e-03
Train: 38 [ 350/390]  Loss: 0.4569 (0.291)  Acc@1: 85.9375 (90.1086)  Acc@5: 100.0000 (99.7997)LR: 4.252e-03
Train: 38 [ 390/390]  Loss: 0.2479 (0.290)  Acc@1: 92.5000 (90.1360)  Acc@5: 100.0000 (99.8120)LR: 4.252e-03
train_acc 90.136000
Valid: 38 [   0/390]  Loss: 0.6450 (0.645)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)
Valid: 38 [  50/390]  Loss: 0.4641 (0.478)  Acc@1: 89.0625 (83.7316)  Acc@5: 98.4375 (99.0196)
Valid: 38 [ 100/390]  Loss: 0.5579 (0.489)  Acc@1: 82.8125 (83.6634)  Acc@5: 96.8750 (98.9635)
Valid: 38 [ 150/390]  Loss: 0.4858 (0.479)  Acc@1: 85.9375 (83.9300)  Acc@5: 98.4375 (99.0066)
Valid: 38 [ 200/390]  Loss: 0.5851 (0.481)  Acc@1: 79.6875 (83.7687)  Acc@5: 100.0000 (99.0050)
Valid: 38 [ 250/390]  Loss: 0.4424 (0.483)  Acc@1: 87.5000 (83.7338)  Acc@5: 100.0000 (98.9791)
Valid: 38 [ 300/390]  Loss: 0.3801 (0.484)  Acc@1: 90.6250 (83.7209)  Acc@5: 100.0000 (98.9306)
Valid: 38 [ 350/390]  Loss: 0.4338 (0.481)  Acc@1: 87.5000 (83.8364)  Acc@5: 100.0000 (98.9628)
Valid: 38 [ 390/390]  Loss: 0.3265 (0.481)  Acc@1: 90.0000 (83.8360)  Acc@5: 100.0000 (98.9480)
valid_acc 83.836000
epoch = 38   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3998, 0.2862, 0.3140],
        [0.4852, 0.2076, 0.3073],
        [0.3785, 0.2942, 0.3274],
        [0.4924, 0.2295, 0.2782],
        [0.4924, 0.2460, 0.2616],
        [0.4076, 0.2632, 0.3292],
        [0.5790, 0.1877, 0.2333],
        [0.5365, 0.2176, 0.2459],
        [0.5918, 0.1852, 0.2230],
        [0.4501, 0.2700, 0.2799],
        [0.5646, 0.2239, 0.2115],
        [0.5617, 0.2476, 0.1907],
        [0.5783, 0.1984, 0.2233],
        [0.6521, 0.1395, 0.2085]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3993, 0.2671, 0.3337],
        [0.2807, 0.3208, 0.3985],
        [0.3702, 0.3101, 0.3197],
        [0.3047, 0.3406, 0.3547],
        [0.3955, 0.2955, 0.3091],
        [0.3652, 0.3276, 0.3072],
        [0.3049, 0.3596, 0.3355],
        [0.3553, 0.2847, 0.3601],
        [0.3974, 0.2814, 0.3211],
        [0.3217, 0.3455, 0.3328],
        [0.2969, 0.3431, 0.3601],
        [0.3534, 0.3026, 0.3441],
        [0.3136, 0.3597, 0.3268],
        [0.3475, 0.2985, 0.3539]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 39 [   0/390]  Loss: 0.1913 (0.191)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [  50/390]  Loss: 0.2063 (0.277)  Acc@1: 90.6250 (89.9203)  Acc@5: 100.0000 (99.7549)LR: 3.754e-03
Train: 39 [ 100/390]  Loss: 0.2725 (0.280)  Acc@1: 89.0625 (89.9443)  Acc@5: 100.0000 (99.7834)LR: 3.754e-03
Train: 39 [ 150/390]  Loss: 0.4513 (0.275)  Acc@1: 89.0625 (90.3767)  Acc@5: 98.4375 (99.7930)LR: 3.754e-03
Train: 39 [ 200/390]  Loss: 0.2462 (0.277)  Acc@1: 92.1875 (90.3685)  Acc@5: 100.0000 (99.7979)LR: 3.754e-03
Train: 39 [ 250/390]  Loss: 0.2301 (0.278)  Acc@1: 90.6250 (90.3884)  Acc@5: 100.0000 (99.7946)LR: 3.754e-03
Train: 39 [ 300/390]  Loss: 0.2195 (0.276)  Acc@1: 95.3125 (90.4745)  Acc@5: 98.4375 (99.7975)LR: 3.754e-03
Train: 39 [ 350/390]  Loss: 0.5398 (0.279)  Acc@1: 79.6875 (90.4069)  Acc@5: 100.0000 (99.8086)LR: 3.754e-03
Train: 39 [ 390/390]  Loss: 0.4128 (0.279)  Acc@1: 85.0000 (90.4080)  Acc@5: 100.0000 (99.8040)LR: 3.754e-03
train_acc 90.408000
Valid: 39 [   0/390]  Loss: 0.6024 (0.602)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)
Valid: 39 [  50/390]  Loss: 0.3388 (0.500)  Acc@1: 89.0625 (83.1495)  Acc@5: 100.0000 (99.2341)
Valid: 39 [ 100/390]  Loss: 0.4608 (0.511)  Acc@1: 81.2500 (82.8589)  Acc@5: 100.0000 (98.9480)
Valid: 39 [ 150/390]  Loss: 0.3880 (0.517)  Acc@1: 89.0625 (82.7090)  Acc@5: 100.0000 (98.9756)
Valid: 39 [ 200/390]  Loss: 0.4644 (0.521)  Acc@1: 82.8125 (82.7192)  Acc@5: 100.0000 (98.8884)
Valid: 39 [ 250/390]  Loss: 0.4807 (0.525)  Acc@1: 84.3750 (82.4390)  Acc@5: 96.8750 (98.8608)
Valid: 39 [ 300/390]  Loss: 0.6041 (0.525)  Acc@1: 76.5625 (82.3142)  Acc@5: 98.4375 (98.8632)
Valid: 39 [ 350/390]  Loss: 0.5486 (0.523)  Acc@1: 78.1250 (82.4030)  Acc@5: 98.4375 (98.8470)
Valid: 39 [ 390/390]  Loss: 0.4858 (0.519)  Acc@1: 80.0000 (82.4640)  Acc@5: 100.0000 (98.8440)
valid_acc 82.464000
epoch = 39   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4054, 0.2828, 0.3118],
        [0.4846, 0.2068, 0.3085],
        [0.3823, 0.2925, 0.3252],
        [0.4962, 0.2276, 0.2762],
        [0.4937, 0.2439, 0.2624],
        [0.4101, 0.2607, 0.3292],
        [0.5791, 0.1857, 0.2353],
        [0.5422, 0.2150, 0.2428],
        [0.5963, 0.1828, 0.2209],
        [0.4560, 0.2671, 0.2769],
        [0.5670, 0.2234, 0.2097],
        [0.5652, 0.2466, 0.1882],
        [0.5845, 0.1969, 0.2186],
        [0.6559, 0.1378, 0.2064]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4011, 0.2674, 0.3315],
        [0.2806, 0.3206, 0.3988],
        [0.3681, 0.3115, 0.3204],
        [0.3027, 0.3400, 0.3574],
        [0.3970, 0.2947, 0.3082],
        [0.3677, 0.3279, 0.3044],
        [0.3053, 0.3612, 0.3334],
        [0.3559, 0.2820, 0.3622],
        [0.3976, 0.2832, 0.3192],
        [0.3209, 0.3467, 0.3324],
        [0.2961, 0.3423, 0.3616],
        [0.3545, 0.3013, 0.3443],
        [0.3127, 0.3597, 0.3276],
        [0.3497, 0.2971, 0.3532]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 40 [   0/390]  Loss: 0.2697 (0.270)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 3.292e-03
Train: 40 [  50/390]  Loss: 0.2161 (0.251)  Acc@1: 95.3125 (91.0846)  Acc@5: 100.0000 (99.8468)LR: 3.292e-03
Train: 40 [ 100/390]  Loss: 0.2942 (0.263)  Acc@1: 89.0625 (90.7797)  Acc@5: 100.0000 (99.7989)LR: 3.292e-03
Train: 40 [ 150/390]  Loss: 0.1801 (0.261)  Acc@1: 95.3125 (90.8113)  Acc@5: 98.4375 (99.8448)LR: 3.292e-03
Train: 40 [ 200/390]  Loss: 0.5001 (0.266)  Acc@1: 79.6875 (90.7416)  Acc@5: 98.4375 (99.7979)LR: 3.292e-03
Train: 40 [ 250/390]  Loss: 0.2161 (0.263)  Acc@1: 90.6250 (90.8927)  Acc@5: 100.0000 (99.8070)LR: 3.292e-03
Train: 40 [ 300/390]  Loss: 0.1931 (0.262)  Acc@1: 95.3125 (90.9832)  Acc@5: 100.0000 (99.8183)LR: 3.292e-03
Train: 40 [ 350/390]  Loss: 0.2723 (0.267)  Acc@1: 92.1875 (90.7986)  Acc@5: 100.0000 (99.8086)LR: 3.292e-03
Train: 40 [ 390/390]  Loss: 0.2676 (0.269)  Acc@1: 90.0000 (90.7440)  Acc@5: 100.0000 (99.8040)LR: 3.292e-03
train_acc 90.744000
Valid: 40 [   0/390]  Loss: 0.4086 (0.409)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 40 [  50/390]  Loss: 0.5414 (0.520)  Acc@1: 78.1250 (82.5980)  Acc@5: 98.4375 (98.7439)
Valid: 40 [ 100/390]  Loss: 0.6197 (0.515)  Acc@1: 81.2500 (82.4103)  Acc@5: 98.4375 (98.9325)
Valid: 40 [ 150/390]  Loss: 0.4164 (0.512)  Acc@1: 85.9375 (82.5538)  Acc@5: 100.0000 (98.9238)
Valid: 40 [ 200/390]  Loss: 0.4315 (0.505)  Acc@1: 82.8125 (82.5715)  Acc@5: 98.4375 (98.9894)
Valid: 40 [ 250/390]  Loss: 0.4025 (0.506)  Acc@1: 85.9375 (82.3954)  Acc@5: 96.8750 (99.0040)
Valid: 40 [ 300/390]  Loss: 0.5473 (0.508)  Acc@1: 78.1250 (82.3245)  Acc@5: 98.4375 (99.0033)
Valid: 40 [ 350/390]  Loss: 0.3742 (0.505)  Acc@1: 87.5000 (82.5766)  Acc@5: 100.0000 (99.0207)
Valid: 40 [ 390/390]  Loss: 0.3657 (0.503)  Acc@1: 90.0000 (82.6200)  Acc@5: 100.0000 (99.0280)
valid_acc 82.620000
epoch = 40   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4085, 0.2804, 0.3111],
        [0.4895, 0.2035, 0.3071],
        [0.3850, 0.2893, 0.3257],
        [0.5025, 0.2250, 0.2725],
        [0.4962, 0.2406, 0.2632],
        [0.4170, 0.2603, 0.3227],
        [0.5848, 0.1826, 0.2327],
        [0.5500, 0.2114, 0.2386],
        [0.6026, 0.1782, 0.2191],
        [0.4577, 0.2651, 0.2772],
        [0.5728, 0.2208, 0.2064],
        [0.5716, 0.2442, 0.1841],
        [0.5910, 0.1952, 0.2138],
        [0.6608, 0.1352, 0.2040]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4031, 0.2684, 0.3285],
        [0.2806, 0.3214, 0.3980],
        [0.3693, 0.3109, 0.3198],
        [0.3023, 0.3391, 0.3586],
        [0.3947, 0.2930, 0.3123],
        [0.3731, 0.3279, 0.2990],
        [0.3049, 0.3620, 0.3331],
        [0.3549, 0.2790, 0.3661],
        [0.3986, 0.2835, 0.3179],
        [0.3197, 0.3455, 0.3348],
        [0.2950, 0.3413, 0.3637],
        [0.3539, 0.3022, 0.3439],
        [0.3099, 0.3603, 0.3298],
        [0.3492, 0.2977, 0.3531]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 41 [   0/390]  Loss: 0.3097 (0.310)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 2.868e-03
Train: 41 [  50/390]  Loss: 0.4482 (0.277)  Acc@1: 87.5000 (90.4412)  Acc@5: 100.0000 (99.8162)LR: 2.868e-03
Train: 41 [ 100/390]  Loss: 0.3936 (0.277)  Acc@1: 87.5000 (89.9907)  Acc@5: 98.4375 (99.8144)LR: 2.868e-03
Train: 41 [ 150/390]  Loss: 0.2278 (0.277)  Acc@1: 87.5000 (90.0559)  Acc@5: 100.0000 (99.8137)LR: 2.868e-03
Train: 41 [ 200/390]  Loss: 0.2833 (0.272)  Acc@1: 90.6250 (90.3141)  Acc@5: 100.0000 (99.8445)LR: 2.868e-03
Train: 41 [ 250/390]  Loss: 0.1255 (0.273)  Acc@1: 96.8750 (90.3386)  Acc@5: 100.0000 (99.8257)LR: 2.868e-03
Train: 41 [ 300/390]  Loss: 0.1866 (0.272)  Acc@1: 95.3125 (90.4848)  Acc@5: 100.0000 (99.8131)LR: 2.868e-03
Train: 41 [ 350/390]  Loss: 0.1495 (0.269)  Acc@1: 95.3125 (90.5582)  Acc@5: 100.0000 (99.8130)LR: 2.868e-03
Train: 41 [ 390/390]  Loss: 0.1705 (0.268)  Acc@1: 97.5000 (90.5840)  Acc@5: 100.0000 (99.8280)LR: 2.868e-03
train_acc 90.584000
Valid: 41 [   0/390]  Loss: 0.4597 (0.460)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 41 [  50/390]  Loss: 0.6543 (0.446)  Acc@1: 76.5625 (84.8958)  Acc@5: 100.0000 (99.4792)
Valid: 41 [ 100/390]  Loss: 0.4450 (0.437)  Acc@1: 84.3750 (85.2723)  Acc@5: 98.4375 (99.4585)
Valid: 41 [ 150/390]  Loss: 0.2053 (0.444)  Acc@1: 96.8750 (85.2339)  Acc@5: 98.4375 (99.2964)
Valid: 41 [ 200/390]  Loss: 0.4299 (0.443)  Acc@1: 89.0625 (85.2146)  Acc@5: 98.4375 (99.2926)
Valid: 41 [ 250/390]  Loss: 0.5223 (0.445)  Acc@1: 82.8125 (85.1656)  Acc@5: 96.8750 (99.2592)
Valid: 41 [ 300/390]  Loss: 0.5260 (0.444)  Acc@1: 85.9375 (85.1329)  Acc@5: 98.4375 (99.2369)
Valid: 41 [ 350/390]  Loss: 0.2796 (0.440)  Acc@1: 93.7500 (85.2475)  Acc@5: 100.0000 (99.2432)
Valid: 41 [ 390/390]  Loss: 0.7462 (0.441)  Acc@1: 77.5000 (85.1920)  Acc@5: 100.0000 (99.2520)
valid_acc 85.192000
epoch = 41   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4110, 0.2788, 0.3102],
        [0.4929, 0.1990, 0.3081],
        [0.3870, 0.2885, 0.3245],
        [0.5083, 0.2207, 0.2710],
        [0.4999, 0.2387, 0.2614],
        [0.4190, 0.2591, 0.3219],
        [0.5906, 0.1775, 0.2319],
        [0.5549, 0.2084, 0.2367],
        [0.6084, 0.1744, 0.2172],
        [0.4581, 0.2647, 0.2772],
        [0.5803, 0.2174, 0.2023],
        [0.5754, 0.2430, 0.1817],
        [0.5960, 0.1935, 0.2105],
        [0.6634, 0.1343, 0.2023]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4075, 0.2689, 0.3235],
        [0.2774, 0.3253, 0.3974],
        [0.3710, 0.3117, 0.3174],
        [0.3014, 0.3390, 0.3596],
        [0.3921, 0.2918, 0.3162],
        [0.3794, 0.3265, 0.2941],
        [0.3036, 0.3624, 0.3341],
        [0.3554, 0.2784, 0.3662],
        [0.3983, 0.2827, 0.3191],
        [0.3200, 0.3439, 0.3361],
        [0.2933, 0.3415, 0.3652],
        [0.3533, 0.3039, 0.3428],
        [0.3082, 0.3608, 0.3310],
        [0.3496, 0.2981, 0.3523]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 42 [   0/390]  Loss: 0.2432 (0.243)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [  50/390]  Loss: 0.2215 (0.254)  Acc@1: 90.6250 (90.8395)  Acc@5: 100.0000 (99.8162)LR: 2.484e-03
Train: 42 [ 100/390]  Loss: 0.3481 (0.259)  Acc@1: 87.5000 (91.0582)  Acc@5: 100.0000 (99.7834)LR: 2.484e-03
Train: 42 [ 150/390]  Loss: 0.4292 (0.263)  Acc@1: 84.3750 (90.9458)  Acc@5: 96.8750 (99.8034)LR: 2.484e-03
Train: 42 [ 200/390]  Loss: 0.2879 (0.263)  Acc@1: 90.6250 (90.9904)  Acc@5: 100.0000 (99.7746)LR: 2.484e-03
Train: 42 [ 250/390]  Loss: 0.1541 (0.264)  Acc@1: 95.3125 (90.8055)  Acc@5: 100.0000 (99.7946)LR: 2.484e-03
Train: 42 [ 300/390]  Loss: 0.2689 (0.265)  Acc@1: 90.6250 (90.8223)  Acc@5: 100.0000 (99.7975)LR: 2.484e-03
Train: 42 [ 350/390]  Loss: 0.2001 (0.266)  Acc@1: 92.1875 (90.7274)  Acc@5: 100.0000 (99.7997)LR: 2.484e-03
Train: 42 [ 390/390]  Loss: 0.2661 (0.265)  Acc@1: 90.0000 (90.7280)  Acc@5: 100.0000 (99.8040)LR: 2.484e-03
train_acc 90.728000
Valid: 42 [   0/390]  Loss: 0.4876 (0.488)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)
Valid: 42 [  50/390]  Loss: 0.4597 (0.543)  Acc@1: 85.9375 (80.9743)  Acc@5: 100.0000 (98.9890)
Valid: 42 [ 100/390]  Loss: 0.3385 (0.547)  Acc@1: 92.1875 (81.1262)  Acc@5: 98.4375 (98.7933)
Valid: 42 [ 150/390]  Loss: 0.4539 (0.543)  Acc@1: 82.8125 (81.4776)  Acc@5: 100.0000 (98.6858)
Valid: 42 [ 200/390]  Loss: 0.5307 (0.544)  Acc@1: 84.3750 (81.6465)  Acc@5: 98.4375 (98.6474)
Valid: 42 [ 250/390]  Loss: 0.5440 (0.548)  Acc@1: 76.5625 (81.6297)  Acc@5: 96.8750 (98.6429)
Valid: 42 [ 300/390]  Loss: 0.5268 (0.541)  Acc@1: 84.3750 (81.8729)  Acc@5: 98.4375 (98.7022)
Valid: 42 [ 350/390]  Loss: 0.4140 (0.542)  Acc@1: 85.9375 (81.7931)  Acc@5: 100.0000 (98.7402)
Valid: 42 [ 390/390]  Loss: 0.5028 (0.540)  Acc@1: 82.5000 (81.8680)  Acc@5: 100.0000 (98.7720)
valid_acc 81.868000
epoch = 42   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4101, 0.2793, 0.3107],
        [0.4969, 0.1962, 0.3069],
        [0.3894, 0.2897, 0.3210],
        [0.5148, 0.2170, 0.2682],
        [0.5020, 0.2381, 0.2599],
        [0.4193, 0.2605, 0.3202],
        [0.5944, 0.1755, 0.2302],
        [0.5597, 0.2070, 0.2333],
        [0.6169, 0.1719, 0.2112],
        [0.4632, 0.2634, 0.2734],
        [0.5848, 0.2161, 0.1991],
        [0.5796, 0.2411, 0.1793],
        [0.6024, 0.1904, 0.2072],
        [0.6689, 0.1327, 0.1984]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4073, 0.2691, 0.3236],
        [0.2777, 0.3254, 0.3969],
        [0.3705, 0.3102, 0.3193],
        [0.3023, 0.3412, 0.3565],
        [0.3944, 0.2904, 0.3151],
        [0.3779, 0.3267, 0.2954],
        [0.3058, 0.3634, 0.3308],
        [0.3568, 0.2786, 0.3646],
        [0.4015, 0.2817, 0.3169],
        [0.3177, 0.3456, 0.3367],
        [0.2925, 0.3411, 0.3664],
        [0.3510, 0.3055, 0.3435],
        [0.3083, 0.3598, 0.3319],
        [0.3499, 0.2960, 0.3541]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 43 [   0/390]  Loss: 0.3585 (0.358)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [  50/390]  Loss: 0.2393 (0.263)  Acc@1: 93.7500 (90.9620)  Acc@5: 100.0000 (99.8468)LR: 2.142e-03
Train: 43 [ 100/390]  Loss: 0.3017 (0.247)  Acc@1: 89.0625 (91.4759)  Acc@5: 98.4375 (99.8608)LR: 2.142e-03
Train: 43 [ 150/390]  Loss: 0.3087 (0.255)  Acc@1: 82.8125 (91.0286)  Acc@5: 100.0000 (99.8758)LR: 2.142e-03
Train: 43 [ 200/390]  Loss: 0.3182 (0.258)  Acc@1: 87.5000 (91.0292)  Acc@5: 100.0000 (99.8678)LR: 2.142e-03
Train: 43 [ 250/390]  Loss: 0.2199 (0.260)  Acc@1: 90.6250 (90.8802)  Acc@5: 100.0000 (99.8381)LR: 2.142e-03
Train: 43 [ 300/390]  Loss: 0.3248 (0.264)  Acc@1: 89.0625 (90.6977)  Acc@5: 100.0000 (99.8495)LR: 2.142e-03
Train: 43 [ 350/390]  Loss: 0.2213 (0.265)  Acc@1: 95.3125 (90.7007)  Acc@5: 100.0000 (99.8442)LR: 2.142e-03
Train: 43 [ 390/390]  Loss: 0.5648 (0.267)  Acc@1: 77.5000 (90.6280)  Acc@5: 100.0000 (99.8480)LR: 2.142e-03
train_acc 90.628000
Valid: 43 [   0/390]  Loss: 0.5228 (0.523)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 43 [  50/390]  Loss: 0.4841 (0.531)  Acc@1: 79.6875 (81.8934)  Acc@5: 100.0000 (98.8358)
Valid: 43 [ 100/390]  Loss: 0.6127 (0.528)  Acc@1: 84.3750 (82.0545)  Acc@5: 96.8750 (98.9171)
Valid: 43 [ 150/390]  Loss: 0.4856 (0.523)  Acc@1: 85.9375 (82.3882)  Acc@5: 96.8750 (98.8514)
Valid: 43 [ 200/390]  Loss: 0.6131 (0.521)  Acc@1: 76.5625 (82.4627)  Acc@5: 100.0000 (98.8728)
Valid: 43 [ 250/390]  Loss: 0.5261 (0.522)  Acc@1: 76.5625 (82.4763)  Acc@5: 100.0000 (98.8733)
Valid: 43 [ 300/390]  Loss: 0.4644 (0.524)  Acc@1: 85.9375 (82.3868)  Acc@5: 100.0000 (98.8995)
Valid: 43 [ 350/390]  Loss: 0.4540 (0.522)  Acc@1: 84.3750 (82.5053)  Acc@5: 98.4375 (98.8916)
Valid: 43 [ 390/390]  Loss: 0.4336 (0.522)  Acc@1: 85.0000 (82.5080)  Acc@5: 100.0000 (98.8960)
valid_acc 82.508000
epoch = 43   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4140, 0.2750, 0.3111],
        [0.4988, 0.1944, 0.3068],
        [0.3910, 0.2872, 0.3218],
        [0.5209, 0.2139, 0.2651],
        [0.5047, 0.2366, 0.2588],
        [0.4238, 0.2584, 0.3179],
        [0.5997, 0.1722, 0.2281],
        [0.5627, 0.2041, 0.2331],
        [0.6218, 0.1687, 0.2095],
        [0.4695, 0.2594, 0.2711],
        [0.5884, 0.2149, 0.1967],
        [0.5814, 0.2397, 0.1789],
        [0.6077, 0.1884, 0.2039],
        [0.6704, 0.1320, 0.1976]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4074, 0.2681, 0.3245],
        [0.2777, 0.3239, 0.3985],
        [0.3710, 0.3100, 0.3190],
        [0.3019, 0.3414, 0.3567],
        [0.3925, 0.2932, 0.3144],
        [0.3820, 0.3248, 0.2932],
        [0.3055, 0.3634, 0.3311],
        [0.3558, 0.2767, 0.3675],
        [0.4019, 0.2832, 0.3149],
        [0.3190, 0.3440, 0.3371],
        [0.2915, 0.3383, 0.3702],
        [0.3507, 0.3054, 0.3439],
        [0.3082, 0.3602, 0.3315],
        [0.3469, 0.2951, 0.3580]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 44 [   0/390]  Loss: 0.1414 (0.141)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.843e-03
Train: 44 [  50/390]  Loss: 0.2189 (0.258)  Acc@1: 87.5000 (91.0233)  Acc@5: 100.0000 (99.7855)LR: 1.843e-03
Train: 44 [ 100/390]  Loss: 0.3739 (0.259)  Acc@1: 84.3750 (90.9653)  Acc@5: 100.0000 (99.7834)LR: 1.843e-03
Train: 44 [ 150/390]  Loss: 0.1443 (0.260)  Acc@1: 95.3125 (90.9147)  Acc@5: 100.0000 (99.7930)LR: 1.843e-03
Train: 44 [ 200/390]  Loss: 0.1823 (0.261)  Acc@1: 95.3125 (90.9515)  Acc@5: 100.0000 (99.8212)LR: 1.843e-03
Train: 44 [ 250/390]  Loss: 0.2187 (0.263)  Acc@1: 90.6250 (90.8055)  Acc@5: 100.0000 (99.8195)LR: 1.843e-03
Train: 44 [ 300/390]  Loss: 0.2138 (0.263)  Acc@1: 90.6250 (90.8171)  Acc@5: 100.0000 (99.8235)LR: 1.843e-03
Train: 44 [ 350/390]  Loss: 0.1303 (0.264)  Acc@1: 98.4375 (90.7853)  Acc@5: 100.0000 (99.8264)LR: 1.843e-03
Train: 44 [ 390/390]  Loss: 0.1241 (0.263)  Acc@1: 97.5000 (90.8520)  Acc@5: 100.0000 (99.8200)LR: 1.843e-03
train_acc 90.852000
Valid: 44 [   0/390]  Loss: 0.5387 (0.539)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 44 [  50/390]  Loss: 0.6209 (0.523)  Acc@1: 81.2500 (82.2917)  Acc@5: 98.4375 (99.0809)
Valid: 44 [ 100/390]  Loss: 0.5383 (0.538)  Acc@1: 84.3750 (81.6986)  Acc@5: 96.8750 (98.8397)
Valid: 44 [ 150/390]  Loss: 0.2932 (0.524)  Acc@1: 89.0625 (82.1916)  Acc@5: 100.0000 (98.8825)
Valid: 44 [ 200/390]  Loss: 0.4831 (0.523)  Acc@1: 82.8125 (82.2994)  Acc@5: 100.0000 (98.9039)
Valid: 44 [ 250/390]  Loss: 0.5933 (0.520)  Acc@1: 81.2500 (82.3207)  Acc@5: 98.4375 (98.9480)
Valid: 44 [ 300/390]  Loss: 0.5187 (0.520)  Acc@1: 84.3750 (82.4336)  Acc@5: 100.0000 (98.9618)
Valid: 44 [ 350/390]  Loss: 0.5796 (0.524)  Acc@1: 78.1250 (82.3495)  Acc@5: 100.0000 (98.9494)
Valid: 44 [ 390/390]  Loss: 0.5707 (0.526)  Acc@1: 82.5000 (82.2320)  Acc@5: 100.0000 (98.9400)
valid_acc 82.232000
epoch = 44   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4172, 0.2715, 0.3112],
        [0.5013, 0.1924, 0.3064],
        [0.3943, 0.2855, 0.3202],
        [0.5233, 0.2125, 0.2643],
        [0.5073, 0.2359, 0.2568],
        [0.4303, 0.2567, 0.3130],
        [0.6031, 0.1709, 0.2261],
        [0.5642, 0.2022, 0.2336],
        [0.6255, 0.1658, 0.2087],
        [0.4733, 0.2560, 0.2707],
        [0.5903, 0.2138, 0.1959],
        [0.5832, 0.2390, 0.1778],
        [0.6143, 0.1852, 0.2006],
        [0.6717, 0.1318, 0.1966]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4103, 0.2678, 0.3219],
        [0.2770, 0.3258, 0.3972],
        [0.3726, 0.3086, 0.3188],
        [0.3003, 0.3394, 0.3603],
        [0.3935, 0.2925, 0.3140],
        [0.3838, 0.3244, 0.2919],
        [0.3051, 0.3651, 0.3298],
        [0.3556, 0.2775, 0.3668],
        [0.4014, 0.2845, 0.3140],
        [0.3211, 0.3418, 0.3371],
        [0.2884, 0.3395, 0.3721],
        [0.3494, 0.3031, 0.3475],
        [0.3077, 0.3592, 0.3331],
        [0.3466, 0.2947, 0.3587]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 45 [   0/390]  Loss: 0.2721 (0.272)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.587e-03
Train: 45 [  50/390]  Loss: 0.1892 (0.237)  Acc@1: 92.1875 (91.7279)  Acc@5: 100.0000 (99.8468)LR: 1.587e-03
Train: 45 [ 100/390]  Loss: 0.3692 (0.249)  Acc@1: 89.0625 (91.5068)  Acc@5: 100.0000 (99.7989)LR: 1.587e-03
Train: 45 [ 150/390]  Loss: 0.2337 (0.253)  Acc@1: 90.6250 (91.5046)  Acc@5: 100.0000 (99.7930)LR: 1.587e-03
Train: 45 [ 200/390]  Loss: 0.3285 (0.252)  Acc@1: 85.9375 (91.4723)  Acc@5: 100.0000 (99.7901)LR: 1.587e-03
Train: 45 [ 250/390]  Loss: 0.5954 (0.255)  Acc@1: 78.1250 (91.3533)  Acc@5: 100.0000 (99.7946)LR: 1.587e-03
Train: 45 [ 300/390]  Loss: 0.2045 (0.257)  Acc@1: 93.7500 (91.2116)  Acc@5: 100.0000 (99.8235)LR: 1.587e-03
Train: 45 [ 350/390]  Loss: 0.2504 (0.261)  Acc@1: 89.0625 (91.0657)  Acc@5: 100.0000 (99.8130)LR: 1.587e-03
Train: 45 [ 390/390]  Loss: 0.1966 (0.261)  Acc@1: 97.5000 (91.0760)  Acc@5: 100.0000 (99.8240)LR: 1.587e-03
train_acc 91.076000
Valid: 45 [   0/390]  Loss: 0.8184 (0.818)  Acc@1: 75.0000 (75.0000)  Acc@5: 95.3125 (95.3125)
Valid: 45 [  50/390]  Loss: 0.6983 (0.590)  Acc@1: 71.8750 (81.2806)  Acc@5: 98.4375 (98.2230)
Valid: 45 [ 100/390]  Loss: 0.6415 (0.582)  Acc@1: 81.2500 (81.2036)  Acc@5: 98.4375 (98.3447)
Valid: 45 [ 150/390]  Loss: 0.3945 (0.578)  Acc@1: 87.5000 (80.8464)  Acc@5: 98.4375 (98.5099)
Valid: 45 [ 200/390]  Loss: 0.7643 (0.584)  Acc@1: 75.0000 (80.4960)  Acc@5: 95.3125 (98.5308)
Valid: 45 [ 250/390]  Loss: 0.8291 (0.585)  Acc@1: 68.7500 (80.3785)  Acc@5: 95.3125 (98.5433)
Valid: 45 [ 300/390]  Loss: 0.5291 (0.586)  Acc@1: 81.2500 (80.2585)  Acc@5: 100.0000 (98.6140)
Valid: 45 [ 350/390]  Loss: 0.6019 (0.586)  Acc@1: 76.5625 (80.2796)  Acc@5: 100.0000 (98.6245)
Valid: 45 [ 390/390]  Loss: 0.8588 (0.580)  Acc@1: 72.5000 (80.4680)  Acc@5: 95.0000 (98.6520)
valid_acc 80.468000
epoch = 45   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('skip_connect', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4189, 0.2677, 0.3134],
        [0.5052, 0.1886, 0.3061],
        [0.3958, 0.2833, 0.3209],
        [0.5282, 0.2090, 0.2627],
        [0.5093, 0.2349, 0.2558],
        [0.4332, 0.2555, 0.3114],
        [0.6088, 0.1667, 0.2245],
        [0.5657, 0.1990, 0.2353],
        [0.6297, 0.1631, 0.2072],
        [0.4745, 0.2536, 0.2719],
        [0.5982, 0.2094, 0.1924],
        [0.5876, 0.2361, 0.1763],
        [0.6206, 0.1817, 0.1978],
        [0.6744, 0.1305, 0.1951]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4118, 0.2664, 0.3218],
        [0.2767, 0.3239, 0.3994],
        [0.3723, 0.3108, 0.3169],
        [0.3008, 0.3408, 0.3583],
        [0.3953, 0.2926, 0.3121],
        [0.3850, 0.3241, 0.2909],
        [0.3028, 0.3685, 0.3287],
        [0.3544, 0.2771, 0.3685],
        [0.4028, 0.2849, 0.3123],
        [0.3206, 0.3418, 0.3375],
        [0.2887, 0.3377, 0.3737],
        [0.3482, 0.3035, 0.3483],
        [0.3074, 0.3603, 0.3323],
        [0.3447, 0.2966, 0.3587]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 46 [   0/390]  Loss: 0.2003 (0.200)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [  50/390]  Loss: 0.1102 (0.265)  Acc@1: 95.3125 (90.9620)  Acc@5: 100.0000 (99.8162)LR: 1.377e-03
Train: 46 [ 100/390]  Loss: 0.3169 (0.257)  Acc@1: 85.9375 (91.0891)  Acc@5: 100.0000 (99.7989)LR: 1.377e-03
Train: 46 [ 150/390]  Loss: 0.3551 (0.257)  Acc@1: 87.5000 (91.0700)  Acc@5: 100.0000 (99.8241)LR: 1.377e-03
Train: 46 [ 200/390]  Loss: 0.2967 (0.260)  Acc@1: 93.7500 (91.1225)  Acc@5: 98.4375 (99.7979)LR: 1.377e-03
Train: 46 [ 250/390]  Loss: 0.1352 (0.263)  Acc@1: 95.3125 (91.0172)  Acc@5: 100.0000 (99.8132)LR: 1.377e-03
Train: 46 [ 300/390]  Loss: 0.3818 (0.264)  Acc@1: 85.9375 (90.9365)  Acc@5: 100.0000 (99.8079)LR: 1.377e-03
Train: 46 [ 350/390]  Loss: 0.1549 (0.261)  Acc@1: 96.8750 (91.0390)  Acc@5: 100.0000 (99.8041)LR: 1.377e-03
Train: 46 [ 390/390]  Loss: 0.2515 (0.261)  Acc@1: 92.5000 (91.0440)  Acc@5: 100.0000 (99.8200)LR: 1.377e-03
train_acc 91.044000
Valid: 46 [   0/390]  Loss: 0.5504 (0.550)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 46 [  50/390]  Loss: 0.5375 (0.631)  Acc@1: 82.8125 (79.3505)  Acc@5: 98.4375 (98.6520)
Valid: 46 [ 100/390]  Loss: 0.7072 (0.632)  Acc@1: 71.8750 (78.7283)  Acc@5: 98.4375 (98.5922)
Valid: 46 [ 150/390]  Loss: 0.6844 (0.636)  Acc@1: 70.3125 (78.5182)  Acc@5: 100.0000 (98.5099)
Valid: 46 [ 200/390]  Loss: 0.5570 (0.633)  Acc@1: 79.6875 (78.6303)  Acc@5: 100.0000 (98.5463)
Valid: 46 [ 250/390]  Loss: 0.5779 (0.641)  Acc@1: 85.9375 (78.4114)  Acc@5: 96.8750 (98.4250)
Valid: 46 [ 300/390]  Loss: 0.8916 (0.645)  Acc@1: 67.1875 (78.2859)  Acc@5: 93.7500 (98.3544)
Valid: 46 [ 350/390]  Loss: 0.4498 (0.642)  Acc@1: 81.2500 (78.3387)  Acc@5: 100.0000 (98.3574)
Valid: 46 [ 390/390]  Loss: 0.6952 (0.645)  Acc@1: 75.0000 (78.2480)  Acc@5: 100.0000 (98.3240)
valid_acc 78.248000
epoch = 46   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4216, 0.2677, 0.3107],
        [0.5028, 0.1882, 0.3090],
        [0.3980, 0.2829, 0.3191],
        [0.5302, 0.2085, 0.2613],
        [0.5084, 0.2350, 0.2567],
        [0.4345, 0.2552, 0.3104],
        [0.6114, 0.1648, 0.2238],
        [0.5662, 0.1978, 0.2360],
        [0.6329, 0.1615, 0.2056],
        [0.4757, 0.2538, 0.2705],
        [0.5998, 0.2106, 0.1896],
        [0.5897, 0.2366, 0.1737],
        [0.6256, 0.1815, 0.1929],
        [0.6765, 0.1313, 0.1921]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4133, 0.2661, 0.3206],
        [0.2758, 0.3220, 0.4022],
        [0.3712, 0.3122, 0.3166],
        [0.3003, 0.3406, 0.3592],
        [0.3960, 0.2942, 0.3098],
        [0.3861, 0.3242, 0.2897],
        [0.3013, 0.3698, 0.3288],
        [0.3522, 0.2756, 0.3722],
        [0.4050, 0.2857, 0.3093],
        [0.3208, 0.3427, 0.3365],
        [0.2880, 0.3379, 0.3741],
        [0.3500, 0.3040, 0.3460],
        [0.3097, 0.3591, 0.3312],
        [0.3454, 0.2944, 0.3601]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 47 [   0/390]  Loss: 0.1953 (0.195)  Acc@1: 93.7500 (93.7500)  Acc@5: 98.4375 (98.4375)LR: 1.213e-03
Train: 47 [  50/390]  Loss: 0.4331 (0.243)  Acc@1: 87.5000 (92.0037)  Acc@5: 98.4375 (99.7855)LR: 1.213e-03
Train: 47 [ 100/390]  Loss: 0.3096 (0.257)  Acc@1: 89.0625 (91.1974)  Acc@5: 100.0000 (99.7679)LR: 1.213e-03
Train: 47 [ 150/390]  Loss: 0.3425 (0.258)  Acc@1: 87.5000 (91.2976)  Acc@5: 98.4375 (99.8241)LR: 1.213e-03
Train: 47 [ 200/390]  Loss: 0.2150 (0.251)  Acc@1: 95.3125 (91.5267)  Acc@5: 100.0000 (99.8212)LR: 1.213e-03
Train: 47 [ 250/390]  Loss: 0.3025 (0.253)  Acc@1: 87.5000 (91.3845)  Acc@5: 100.0000 (99.8319)LR: 1.213e-03
Train: 47 [ 300/390]  Loss: 0.2316 (0.255)  Acc@1: 90.6250 (91.2895)  Acc@5: 98.4375 (99.8339)LR: 1.213e-03
Train: 47 [ 350/390]  Loss: 0.1818 (0.255)  Acc@1: 95.3125 (91.2438)  Acc@5: 100.0000 (99.8308)LR: 1.213e-03
Train: 47 [ 390/390]  Loss: 0.1816 (0.255)  Acc@1: 95.0000 (91.2960)  Acc@5: 100.0000 (99.8320)LR: 1.213e-03
train_acc 91.296000
Valid: 47 [   0/390]  Loss: 0.6156 (0.616)  Acc@1: 79.6875 (79.6875)  Acc@5: 96.8750 (96.8750)
Valid: 47 [  50/390]  Loss: 0.5212 (0.618)  Acc@1: 81.2500 (79.4118)  Acc@5: 100.0000 (98.5294)
Valid: 47 [ 100/390]  Loss: 0.5761 (0.611)  Acc@1: 78.1250 (79.4245)  Acc@5: 100.0000 (98.7005)
Valid: 47 [ 150/390]  Loss: 0.8980 (0.604)  Acc@1: 68.7500 (79.3357)  Acc@5: 96.8750 (98.6134)
Valid: 47 [ 200/390]  Loss: 0.6979 (0.614)  Acc@1: 73.4375 (79.0345)  Acc@5: 100.0000 (98.5697)
Valid: 47 [ 250/390]  Loss: 0.5872 (0.623)  Acc@1: 79.6875 (78.8533)  Acc@5: 98.4375 (98.4624)
Valid: 47 [ 300/390]  Loss: 0.6906 (0.621)  Acc@1: 73.4375 (78.7843)  Acc@5: 93.7500 (98.4271)
Valid: 47 [ 350/390]  Loss: 0.5559 (0.620)  Acc@1: 78.1250 (78.9396)  Acc@5: 98.4375 (98.3752)
Valid: 47 [ 390/390]  Loss: 0.9021 (0.618)  Acc@1: 60.0000 (79.0920)  Acc@5: 100.0000 (98.4040)
valid_acc 79.092000
epoch = 47   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4251, 0.2645, 0.3104],
        [0.5019, 0.1862, 0.3119],
        [0.4026, 0.2800, 0.3174],
        [0.5302, 0.2073, 0.2624],
        [0.5101, 0.2334, 0.2565],
        [0.4364, 0.2544, 0.3092],
        [0.6148, 0.1625, 0.2227],
        [0.5661, 0.1963, 0.2376],
        [0.6379, 0.1600, 0.2020],
        [0.4794, 0.2528, 0.2679],
        [0.6029, 0.2104, 0.1867],
        [0.5910, 0.2365, 0.1725],
        [0.6268, 0.1807, 0.1925],
        [0.6755, 0.1315, 0.1929]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4113, 0.2652, 0.3235],
        [0.2775, 0.3217, 0.4009],
        [0.3705, 0.3146, 0.3149],
        [0.3010, 0.3370, 0.3620],
        [0.3966, 0.2936, 0.3098],
        [0.3865, 0.3231, 0.2904],
        [0.3024, 0.3707, 0.3269],
        [0.3495, 0.2747, 0.3758],
        [0.4043, 0.2873, 0.3084],
        [0.3205, 0.3446, 0.3349],
        [0.2902, 0.3397, 0.3701],
        [0.3509, 0.3029, 0.3462],
        [0.3107, 0.3599, 0.3294],
        [0.3435, 0.2912, 0.3653]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 48 [   0/390]  Loss: 0.2542 (0.254)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [  50/390]  Loss: 0.3185 (0.249)  Acc@1: 87.5000 (90.9926)  Acc@5: 100.0000 (99.8162)LR: 1.095e-03
Train: 48 [ 100/390]  Loss: 0.2544 (0.260)  Acc@1: 90.6250 (90.7642)  Acc@5: 100.0000 (99.8453)LR: 1.095e-03
Train: 48 [ 150/390]  Loss: 0.2363 (0.263)  Acc@1: 93.7500 (90.7699)  Acc@5: 100.0000 (99.8034)LR: 1.095e-03
Train: 48 [ 200/390]  Loss: 0.2318 (0.256)  Acc@1: 92.1875 (91.0448)  Acc@5: 100.0000 (99.7901)LR: 1.095e-03
Train: 48 [ 250/390]  Loss: 0.2716 (0.258)  Acc@1: 90.6250 (91.0608)  Acc@5: 100.0000 (99.7946)LR: 1.095e-03
Train: 48 [ 300/390]  Loss: 0.2388 (0.258)  Acc@1: 93.7500 (91.0766)  Acc@5: 98.4375 (99.7820)LR: 1.095e-03
Train: 48 [ 350/390]  Loss: 0.5276 (0.259)  Acc@1: 84.3750 (91.0123)  Acc@5: 100.0000 (99.7908)LR: 1.095e-03
Train: 48 [ 390/390]  Loss: 0.2308 (0.259)  Acc@1: 87.5000 (91.0080)  Acc@5: 100.0000 (99.7960)LR: 1.095e-03
train_acc 91.008000
Valid: 48 [   0/390]  Loss: 0.5727 (0.573)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)
Valid: 48 [  50/390]  Loss: 0.7755 (0.618)  Acc@1: 73.4375 (78.9828)  Acc@5: 98.4375 (98.5907)
Valid: 48 [ 100/390]  Loss: 0.5630 (0.619)  Acc@1: 78.1250 (78.6200)  Acc@5: 100.0000 (98.5613)
Valid: 48 [ 150/390]  Loss: 0.7628 (0.612)  Acc@1: 73.4375 (78.8700)  Acc@5: 100.0000 (98.6445)
Valid: 48 [ 200/390]  Loss: 0.7079 (0.612)  Acc@1: 73.4375 (78.8091)  Acc@5: 100.0000 (98.7018)
Valid: 48 [ 250/390]  Loss: 0.6077 (0.610)  Acc@1: 78.1250 (78.9592)  Acc@5: 96.8750 (98.7301)
Valid: 48 [ 300/390]  Loss: 0.4845 (0.613)  Acc@1: 85.9375 (78.7635)  Acc@5: 100.0000 (98.7126)
Valid: 48 [ 350/390]  Loss: 0.7403 (0.616)  Acc@1: 73.4375 (78.6013)  Acc@5: 98.4375 (98.7001)
Valid: 48 [ 390/390]  Loss: 0.4802 (0.615)  Acc@1: 82.5000 (78.5600)  Acc@5: 97.5000 (98.6960)
valid_acc 78.560000
epoch = 48   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4271, 0.2619, 0.3110],
        [0.5003, 0.1845, 0.3152],
        [0.4065, 0.2790, 0.3146],
        [0.5323, 0.2059, 0.2619],
        [0.5100, 0.2332, 0.2568],
        [0.4388, 0.2528, 0.3084],
        [0.6166, 0.1612, 0.2222],
        [0.5666, 0.1953, 0.2382],
        [0.6404, 0.1580, 0.2016],
        [0.4825, 0.2526, 0.2649],
        [0.6033, 0.2111, 0.1856],
        [0.5909, 0.2379, 0.1711],
        [0.6271, 0.1812, 0.1917],
        [0.6747, 0.1326, 0.1927]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4116, 0.2638, 0.3246],
        [0.2778, 0.3200, 0.4022],
        [0.3703, 0.3171, 0.3126],
        [0.3012, 0.3329, 0.3659],
        [0.3962, 0.2925, 0.3113],
        [0.3894, 0.3214, 0.2892],
        [0.3022, 0.3727, 0.3251],
        [0.3491, 0.2748, 0.3762],
        [0.4062, 0.2854, 0.3084],
        [0.3207, 0.3463, 0.3331],
        [0.2904, 0.3395, 0.3702],
        [0.3495, 0.3039, 0.3466],
        [0.3099, 0.3591, 0.3310],
        [0.3417, 0.2912, 0.3671]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 49 [   0/390]  Loss: 0.2413 (0.241)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [  50/390]  Loss: 0.1775 (0.275)  Acc@1: 92.1875 (91.1458)  Acc@5: 100.0000 (99.8162)LR: 1.024e-03
Train: 49 [ 100/390]  Loss: 0.1522 (0.248)  Acc@1: 92.1875 (91.8162)  Acc@5: 100.0000 (99.8608)LR: 1.024e-03
Train: 49 [ 150/390]  Loss: 0.2367 (0.252)  Acc@1: 92.1875 (91.6494)  Acc@5: 100.0000 (99.8862)LR: 1.024e-03
Train: 49 [ 200/390]  Loss: 0.1313 (0.252)  Acc@1: 93.7500 (91.5656)  Acc@5: 100.0000 (99.8445)LR: 1.024e-03
Train: 49 [ 250/390]  Loss: 0.2745 (0.252)  Acc@1: 87.5000 (91.4405)  Acc@5: 100.0000 (99.8319)LR: 1.024e-03
Train: 49 [ 300/390]  Loss: 0.1965 (0.254)  Acc@1: 90.6250 (91.2687)  Acc@5: 100.0000 (99.8339)LR: 1.024e-03
Train: 49 [ 350/390]  Loss: 0.1749 (0.255)  Acc@1: 95.3125 (91.2349)  Acc@5: 100.0000 (99.8219)LR: 1.024e-03
Train: 49 [ 390/390]  Loss: 0.3133 (0.255)  Acc@1: 85.0000 (91.2040)  Acc@5: 100.0000 (99.8360)LR: 1.024e-03
train_acc 91.204000
Valid: 49 [   0/390]  Loss: 0.3744 (0.374)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)
Valid: 49 [  50/390]  Loss: 0.6906 (0.600)  Acc@1: 78.1250 (79.4118)  Acc@5: 98.4375 (98.4681)
Valid: 49 [ 100/390]  Loss: 0.5911 (0.600)  Acc@1: 81.2500 (79.0377)  Acc@5: 96.8750 (98.4220)
Valid: 49 [ 150/390]  Loss: 0.5419 (0.592)  Acc@1: 76.5625 (79.5737)  Acc@5: 100.0000 (98.5720)
Valid: 49 [ 200/390]  Loss: 0.7228 (0.594)  Acc@1: 75.0000 (79.6720)  Acc@5: 96.8750 (98.5541)
Valid: 49 [ 250/390]  Loss: 0.5446 (0.598)  Acc@1: 84.3750 (79.5754)  Acc@5: 96.8750 (98.5122)
Valid: 49 [ 300/390]  Loss: 0.5669 (0.597)  Acc@1: 89.0625 (79.8121)  Acc@5: 98.4375 (98.4790)
Valid: 49 [ 350/390]  Loss: 0.6199 (0.596)  Acc@1: 73.4375 (79.8032)  Acc@5: 95.3125 (98.5132)
Valid: 49 [ 390/390]  Loss: 0.6994 (0.596)  Acc@1: 70.0000 (79.8360)  Acc@5: 100.0000 (98.5160)
valid_acc 79.836000
epoch = 49   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4291, 0.2603, 0.3106],
        [0.4996, 0.1820, 0.3183],
        [0.4070, 0.2764, 0.3166],
        [0.5362, 0.2039, 0.2599],
        [0.5094, 0.2332, 0.2574],
        [0.4412, 0.2522, 0.3066],
        [0.6193, 0.1589, 0.2218],
        [0.5656, 0.1941, 0.2404],
        [0.6434, 0.1569, 0.1997],
        [0.4865, 0.2515, 0.2620],
        [0.6093, 0.2079, 0.1828],
        [0.5920, 0.2370, 0.1710],
        [0.6300, 0.1796, 0.1904],
        [0.6751, 0.1323, 0.1926]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4134, 0.2623, 0.3243],
        [0.2773, 0.3189, 0.4039],
        [0.3706, 0.3197, 0.3097],
        [0.3040, 0.3293, 0.3668],
        [0.3971, 0.2912, 0.3117],
        [0.3896, 0.3196, 0.2907],
        [0.3029, 0.3729, 0.3242],
        [0.3498, 0.2734, 0.3768],
        [0.4084, 0.2847, 0.3069],
        [0.3188, 0.3470, 0.3342],
        [0.2903, 0.3369, 0.3728],
        [0.3481, 0.3024, 0.3495],
        [0.3083, 0.3597, 0.3319],
        [0.3398, 0.2907, 0.3696]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
